NPS-OR-04-008 


NAVAL 

POSTGRADUATE 

SCHOOL 


MONTEREY,  CALIFORNIA 


Human  Systems  Integration  and  Automation 

Issues  in 

Small  Unmanned  Aerial  Vehicles 

By 

Michael  E.  McCauley,  Ph.D. 

LT  Panagiotis  Matsangas,  Hellenic  Navy 
October  2004 


Approved  for  public  release;  distribution  unlimited. 

Prepared  for:  Army/NASA  Rotorcraft  Division, 
Aeroflightdynamics  Directorate, 

NASA  Ames  Research  Center, 

Moffett  Field,  CA  94305 


NAVAL  POSTGRADUATE  SCHOOL 
MONTEREY,  CA  93943-5001 


RDML  Patrick  W.  Dunne,  USN  Richard  Elster 

Superintendent  Provost 


This  report  was  prepared  for  and  funded  by  Army/NASA  Rotorcraft  Division, 
Aeroflightdynamics  Directorate,  NASA  Ames  Research  Center,  Moffett  Field,  CA  94305. 

Reproduction  of  all  or  part  of  this  report  is  authorized. 

This  report  was  prepared  by: 


MICHAEL  McCAULEY,  Ph.D.  PANAGIOTIS  MATSANGAS 

Research  Professor  of  LT,  Hellenic  Navy 

Operations  Research 


Reviewed  by: 


LYN  R.  WHITAKER  Released  by: 

Associate  Chairman  for  Research 
Department  of  Operations  Research 


JAMES  N.  EAGLE 
Chairman 

Department  of  Operations  Research 


LEONARD  A.  FERRARI,  Ph.D. 

Associate  Provost  and  Dean  of  Research 


|  REPORT  DOCUMENTATION  PAGE 

|  Form  Approved  OMB  No.  0704-0188  1 

Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response, 
including  the  time  for  reviewing  instruction,  searching  existing  data  sources,  gathering  and  maintaining 
the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments 
regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including 
suggestions  for  reducing  this  burden,  to  Washington  headquarters  Services,  Directorate  for  Information 
Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302,  and  to 
the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188)  Washington  DC 
20503. 

1.  AGENCY  USE  ONLY  (Leave  blank) 

2.  REPORT  DATE 

October  2004 

3.  REPORT  TYPE  AND  DATES 

COVERED 

Technical  Report 

4.  TITLE  AND  SUBTITLE:  Human  Systems  Integration  and  Automation 
Issues  in  Small  Unmanned  Aerial  Vehicles 

5.  FUNDING  NUMBERS 

NPS  RORBP 

6.  AUTHORS:  Michael  E.  McCauley,  Ph.D.;  LT  Panagiotis  Matsangas, 
Hellenic  Navy 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS: 

Naval  Postgraduate  School,  Operations  Research  Department, 

Monterey,  CA  93943 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER: 

NPS-OR-04-008 

9.  SPONSORING  /MONITORING  AGENCY  NAME(S)  AND  ADDRESS: 

Army /  NASA  Rotorcraft  Division,  Aeroflightdynamics  Directorate,  NASA 
Ames  Research  Center,  Moffett  Field,  CA  94305 

10.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER: 

1 1 .  SUPPLEMENTARY  NOTES:  The  views  expressed  in  this  report  are  those  of  the  authors  and  do  not  reflect  the  official 
policy  or  position  of  the  Department  of  Defense  or  the  U.S.  Government. 

12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 

12b.  DISTRIBUTION  CODE 

13.  ABSTRACT  (maximum  200  words) 

The  goal  of  this  report  is  to  identify  Human  System  Integration  (HSI)  and  automation  issues  that  contribute  to 
improved  effectiveness  and  efficiency  in  the  operation  of  U.S.  military  Small  Unmanned  Aerial  Vehicles  (SUAVs). 

HSI  issues  relevant  to  SUAV  operations  are  reviewed  and  observations  from  field  trials  are  summarized. 

Short-term  improvements  are  suggested,  research  issues  are  identified,  and  an  overview  is  provided  of 
automation  technologies  applicable  to  future  SUAV  design. 

14.  SUBJECT  TERMS  Unmanned  Aerial  Vehicle,  Human  Systems  Integration, 
Human-Robot  Interface,  Supervisory  Control,  Mission  Planning  Systems, 

Human  Interface  Design,  Semi-Autonomous  Systems 

15.  NUMBER  OF 
PAGES 

60 

16.  PRICE  CODE 

17.  SECURITY 
CLASSIFICATION  OF 
REPORT 

Unclassified 

18.  SECURITY 
CLASSIFICATION  OF  THIS 
PAGE 

Unclassified 

19.  SECURITY 
CLASSIFICATION  OF 
ABSTRACT 

Unclassified 

20.  LIMITATION  OF 
ABSTRACT 

UL 

NSN  7540-01-280-5500  Standard  Form  298  (Rev.  2-89) 


Prescribed  by  ANSI  Std.  239-18 


ii 


Human  Systems  Integration  and  Automation  Issues  in  SUAVs 


TABLE  OF  CONTENTS 

LIST  OF  TABLES . iv 

LIST  OF  FIGURES . v 

LIST  OF  ACRONYMS . vi 

INTRODUCTION . 1 

Unmanned  Aerial  Vehicle  Definitions  and  History . 1 

Report  Objectives . 3 

SUAV  Overview . 4 

UAV  System  Example . 5 

PROBLEM  DEFINITION . 7 

Manpower  Intensive . 7 

SUAV  Footprint . 8 

Visual  Data  Display  and  Search  Effectiveness . 8 

UAV  Loss  Rate . 12 

HSI  ISSUES  FROM  NEAR-TERM  AND  LONG-TERM  PERSPECTIVES . 16 

Near-T erm  Perspective . 16 

Human  Roles,  Responsibilities,  and  Level  of  Automation . 16 

Command  and  Control  and  Operations . 1 8 

Manning,  Selection,  Training,  and  Fatigue . 19 

Difficult  Operational  Environments . 20 

Procedures  and  Job  Performance  Aids  (JPAs) . 21 

Moving  Control  Platforms . 21 

LONGER-TERM  PERSPECTIVE . 23 

Automation . 23 

Allocation  of  System  Functions  to  Automation  or  Humans . 24 

HSI  Issues  in  Automation . 28 

RESEARCH  RECOMMENDATIONS . 31 

Automation . 31 

Manpower,  Personnel,  and  Training . 31 

Mishaps  and  Mishap  Reporting . 32 

HSI  and  Systems  Analysis . 32 

Policy  and  Procedures . 32 

Human  Factors  Engineering  of  System  or  Subsystem  Design . 32 

Research  Recommendations  from  Appendix  A . 33 

APPENDIX  A: . 34 

AUTONOMY  TECHNOLOGY . 34 

Robotics,  Intelligent  Agents,  and  Multiagent  Systems . 34 

Autonomy . 37 

Swarms  and  Stigmergy . 38 

Semiautonomy  and  the  Human-Robot  Interface . 42 

REFERENCES . 45 


Human  Systems  Integration  and  Automation  Issues  in  SUAVs 


LIST  OF  TABLES 

Table  1.  Four  UAV  Reliability  Metrics  (derived  from  DoD,  2002) . 12 

Table  2.  Example  of  Function/Task  Allocation  for  SUAV  Systems . 25 

Table  3.  Sheridan’s  “Scale  of  Degrees  of  Automation”  (from  Sheridan,  1992) . 27 


Human  Systems  Integration  and  Automation  Issues  in  SUAVs 


LIST  OF  FIGURES 

Figure  1.  Ground  Control  Station . 5 

Figure  2.  Schematic  diagram  of  a  UAV  system . 6 

Figure  3.  Available  time  to  detect  a  target . 9 

Figure  4.  Levels  of  automation  for  each  of  four  stages  of  human  information  processing 

(from  Parasuraman  et  al.,  2000) . 26 

Figure  5.  Control  types  of  multiple  UAVs  (the  thicker  the  communicating  lines,  the  heavier 
the  workload  and  cognitive  demands  on  the  CGS  operators) . 42 


v 


Human  Systems  Integration  and  Automation  Issues  in  SUAVs 


AUVSI 

AVO 

CIRPAS 

DARPA 

DERA 

DoD 

EO/IR 

FOV 

GPS 

GCS 

HALE 

HMD 

HSIAC 

HMMWV 

HRI 

HSI 

ISR 

JPA 

MAS 

MICA 

MPO 

MTBF 

NATO 

ONR 

PDA 

PTZ 

SA 

SI 

SUAV 

UAV 

UCLA 

UV 


LIST  OF  ACRONYMS 

Association  for  Unmanned  Vehicle  Systems  International 
Air  Vehicle  Operator 

Center  for  Interdisciplinary  Remotely  Piloted  Aircraft  Studies 

Defense  Advance  Research  Project  Agency 

Defence  Evaluation  Research  Agency 

Department  of  Defense 

Electro  Optic/  Infra  Red 

Field  of  View 

Global  Positioning  System 

Ground  Control  Station 

High  Altitude  Long  Endurance 

Head-Mounted  Display 

Human-Systems  Information  Analysis  Center 

High  Mobility  Multipurpose  Wheeled  Vehicle 

Human-Robot  Interface 

Human  Systems  Integration 

Intelligence,  Surveillance,  Reconnaissance 

Job  Performance  Aid 

Multi  Agent  Systems 

Mixed  Initiative  Control  of  Automa-teams 

Mission  Payload  Operator 

Mean  Time  Between  Failures 

North  Atlantic  Treaty  Organization 

Office  of  Naval  Research 

Personal  Digital  Assistant 

Pan,  Tilt,  Zoom 

Situational  Awareness 

Swarm  Intelligence 

Small  Unmanned  Aerial  Vehicle 

Unmanned  Aerial  Vehicle 

University  of  California  Los  Angeles 

Unmanned  Vehicles 


VI 


Human  Systems  Integration  and  Automation  Issues  in  SUAVs 


INTRODUCTION 

Unmanned  Aerial  Vehicle  Definitions  and  History 

A  Department  of  Defense  (DoD)  Website 

(http://www.defense.qov/specials/uav2002/uavpaqe01.html)  defined  Unmanned  Aerial 
Vehicles  (UAVs)  as  “powered  aerial  vehicles  sustained  in  flight  by  aerodynamic  lift  over 
most  of  their  flight  path  and  guided  without  an  onboard  crew.”  A  Human  Systems 
Integration  (HSI)  perspective  of  UAV  design  and  operations  is  advocated  in  this  report 
as  the  best  approach  to  improve  UAV  effectiveness  and  to  develop  future 
semiautomated  UAV  systems. 

Research  and  development  of  UAVs  has  been  ongoing  for  at  least  50  years  (Gossett, 
2004)  and  some  sources  claim  that  the  U.S.  military  has  been  involved  with  UAVs  since 
1917  (DoD,  2002).  Operational  UAV  systems  have  proliferated  in  the  past  decade, 
leading  to  the  common  perception  that  UAVs  are  “new  technology.”  Approximately  200 
types  of  UAVs  are  currently  in  the  U.S.  military  inventory  and  the  number  may  rise  to 
500  over  the  next  five  years  (Garamone,  2002).  They  range  from  micro-UAVs  (spanning 
six  to  nine  inches)  to  High-Altitude  Long-Endurance  (HALE)  UAVs  such  as  the  Global 
Hawk,  which  is  approximately  the  same  size  as  a  fighter  aircraft,  operates  at  altitudes  of 
approximately  60,000  feet,  and  has  an  endurance  of  nearly  two  days. 

UAVs  can  provide  the  following  benefits  in  DoD  applications: 

•  Eliminate  the  risk  to  ground  troops  and  aircrew  who  would  otherwise  be 
performing  the  mission; 

•  Enhance  aerodynamic  performance  over  manned  flight  due  to  lighter  weight  and 
freedom  from  human  G-tolerance  constraints;  and 

•  Reduce  cost. 

The  U.S.  military  is  committed  to  rapid  development  and  deployment  of  unmanned 
systems  as  evidenced  by  the  following  goals  stated  by  the  Senate  Armed  Services 
Committee: 
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It  shall  be  a  goal  of  the  Armed  Forces  to  achieve  the  fielding  of 
unmanned,  remotely  controlled  technology  such  that: 

(1)  by  2010,  one  third  of  the  operational  deep  strike  aircraft  of  the  Armed 
Forces  are  unmanned;  and 

(2)  by  201 5,  one  third  of  the  operational  ground  combat  vehicles  of  the 
Armed  Forces  are  unmanned  (Senate  Armed  Services  Committee,  2000). 


An  overview  of  UAV  military  operations  and  development  challenges  can  be  found  in  the 
document,  “DoD  UAV  Roadmap  2002”  (http://www.acq.osd.mil/usd/uav  roadmap.pdf). 
There  has  been  controversy  about  the  basis  for  classifying  or  categorizing  UAVs.  Some 
of  the  suggested  criteria  are  airframe  size,  mission,  support  requirements,  and  cost. 
Airframe  size  is  the  most  commonly  used  criterion.  Small  UAVs  (SUAVs)  are  not 
distinguished  by  unique  function,  mission  relevance,  or  because  they  are  less 
expensive.  The  relevance  of  SUAVs  is  based  on  the  operational  impact  of  their 
logistics — they  offer  greater  flexibility  in  operational  employment  compared  to  larger, 
more  complex  UAV  systems  (DoD,  2002).  The  definition  of  “Small  UAV”  given  in  the 
DoD  UAV  Roadmap  2002  is: 


a.  For  UAVs  designed  to  be  employed  by  themselves  -  any  UAV  system 
where  all  system  components  (i.e.,  air  vehicles,  ground  control/user 
interface  element,  and  communication  equipment)  are  fully  transportable 
by  foot-mobile  troops 

b.  For  UAVs  designed  to  be  employed  from  larger  aircraft  (manned  or 
unmanned)  -  any  UAV  system  where  the  air  vehicle  can  be  loaded  on  the 
larger  aircraft  without  the  use  of  mechanical  loaders  (i.e.,  two-man  lift, 
etc.) 


The  U.S.  Army,  Navy,  Marine  Corps,  and  Special  Operations  Forces  have  invested  in 
the  development  of  SUAVs  to  provide  tactical  imagery  to  small  units  (battalion  and 
below)  for  reconnaissance,  surveillance,  target  acquisition  (RSTA),  and  battle  damage 
assessment  (BDA).  Two  U.S.  Army  UAV  programs,  the  Hunter  and  the  Shadow,  have 
been  deployed  successfully  in  the  Persian  Gulf,  Bosnia,  Afghanistan,  Iraq,  and  other 
conflict  locations  (DoD,  2002). 


The  Army’s  Future  Combat  System  (FCS)  initiative  identifies  a  requirement  for  two 
classes  of  SUAVs: 
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•  Class  I  =  Backpackable;  and 

•  Class  II  =  Vehicle/Man  Transportable. 

In  2003,  the  Army  Strategic  Planning  Board  designated  the  SUAV  as  an  “Urgent 
Wartime  Requirement.”  The  Board  projected  that  SUAVs  will  be  used  in  direct  support 
of  the  Global  War  on  Terrorism  with  immediate  application  to  Army  forces  engaged  in 
Operation  Iraqi  Freedom  and  Operation  Enduring  Freedom. 

UAV  research,  development,  and  acquisition  programs  in  the  DoD  have  been  sporadic 
and  haphazard  over  the  years  and,  perhaps  as  a  consequence,  the  current  technologies 
suffer  from  high  loss  rates  and  require  substantial  manpower  to  operate  and  maintain. 
One  analysis  concluded  that  the  U.S.  has  had  a  “three-decade-long  history  of  poor 
outcomes  in  unmanned  aerial  vehicle  development  efforts”  (Leonard  and  Drezner, 

2002).  However,  the  U.S.  is  entering  a  new  era  in  which  the  unbridled  engineering 
innovation  that  has  characterized  UAV  design  and  development,  is  likely  to  give  way  to 
more  systematic,  systems  engineering  approaches. 

Report  Objectives 

The  goal  of  this  report  is  to  identify  HSI  and  automation  issues  that  contribute  to 
improved  effectiveness,  efficiency,  and  risk  management  in  the  operation  of  U.S.  military 
SUAVs.  To  that  end,  the  following  three  objectives  were  defined: 

1 )  Summarize  current  HSI  issues  relevant  to  SUAV  operations; 

2)  Identify  areas  where  short-term  HSI  improvements  might  yield  gains  in 
effectiveness;  and 

3)  Identify  research,  design  concepts,  and  technologies  in  automation  and  related 
disciplines  that  are  applicable  to  future  SUAVs. 

SUAVs  hold  great  promise  and  will  evolve  into  valuable  tactical  assets.  Any  issues  or 
problems  identified  in  this  report  are  intended  to  hasten  the  improvement  cycle. 
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The  present  report  is  based  on  three  primary  sources  of  information: 

1)  Participation  in  a  series  of  UAV  experiments  and  field  trials  at  Camp  Roberts, 
California,  conducted  by  the  Naval  Postgraduate  School  under  the  direction  of 
Dr.  David  Netzer; 

2)  Review  of  the  literature  in  UAVs,  semiautonomous  systems,  robotics,  and  related 
areas,  with  a  focus  on  HSI  and  evolving  technology;  and 

3)  Attendance  at  the  Association  for  Unmanned  Vehicle  Systems  International 
(AUVSI)  2003  conference  and  flight  demonstrations  at  Webster  Field,  Maryland. 

SUAV  Overview 

This  paper  will  focus  on  U.S.  military  SUAVs,  as  opposed  to  HALE  UAVs  or  Medium 
Altitude  and  Endurance  (MAE)  UAVs,  such  as  the  Global  Hawk  or  the  Predator.  High 
altitude  generally  refers  to  above  50,000  feet  and  long  endurance  to  24  hours  or  more. 
By  contrast,  current  SUAVs  tend  to  operate  at  less  than  5,000  feet  and  their  endurance 
is  on  the  order  of  several  hours.  Examples  of  current  SUAVs  are  the  TERN,  Silver  Fox, 
Swift,  Pointer,  Raven,  and  Dragon  Eye.  The  Hunter  and  the  Shadow,  by  contrast,  are 
larger  and  usually  are  called  “Tactical”  UAVs  (TUAVs).  In  this  report,  the  term  SUAV  is 
used  routinely,  but  the  content  of  this  report  is  equally  appropriate  for  both  SUAV  and 
TUAV  design  concepts  and  operation. 

SUAV  missions  often  are  described  as  “what’s  over  the  next  hill?”  Related  questions 
might  be,  “where  are  they?”  or  “how  many  are  there?”  SUAVs  enable  small,  ground- 
based  military  units  to  perform  intelligence,  surveillance,  and  reconnaissance  (ISR) 
missions,  reducing  the  risk  to  mounted  or  dismounted  troops  who  would  otherwise 
perform  the  ISR  mission.  The  objective  is  to  extend  the  “eyes”  of  the  unit  over  a  greater 
range,  more  quickly,  and  with  reduced  risk  to  human  life.  According  to  the  DoD 
Roadmap  (2002),  one  of  the  five  “historically  validated  UAV  roles  is  small  unit  asset  for 
over-the-hill  reconnaissance.”  SUAVs  will  be  deployable  at  or  near  the  front  lines,  at  the 
company  or  platoon  level,  and  “will  provide  the  commander  with  what  amounts  to  a  pair 
of  flying  binoculars”  (DoD,  2002). 

SUAVs  provide  electro-optical  (EO)  or  infrared  (IR)  sensor  data  leading  to  the  detection, 
classification,  and  identification  of  vehicles,  people,  and  other  tactically  relevant  objects. 
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From  the  HSI  perspective,  it  is  important  to  note  that  SUAVs  do  not  detect,  classify,  or 
identify  anything — people  do.  The  SUAV  provides  the  video  or  IR  imagery  or  other  data 
that  enables  properly  trained  observers  to  perform  the  detection,  classification,  and 
identification  functions. 

UAV System  Example 

The  basic  structure  of  a  typical  UAV  system  may  be  viewed  as  an  aircraft  with  the  crew 
physically  remote  from  the  vehicle.  The  upside  of  the  remote  crew  is  reduced  human 
risk,  the  downside  is  reduced  sensory  input  about  the  status  of  the  aircraft  (e.g.,  no 
noise,  vibration,  vestibular,  or  proprioceptive  input)  and  reduced  visual  information 
regarding  terrain,  weather,  air  traffic,  and  threats. 

A  typical  UAV  system  has  a  minimum  of  two  operators,  a  pilot  or  Air  Vehicle  Operator 
(AVO)  and  a  sensor  operator  or  Mission  Payload  Operator  (MPO).  As  shown  in  Figure 
1,  these  operators  work  in  the  Ground  Control  Station  (GCS),  which  typically  is  installed 
in  a  stationary  High  Mobility  Multipurpose  Wheeled  Vehicle  (HMMWV;  or“Humvee”). 


Figure  1.  Ground  Control  Station. 

A  schematic  diagram  of  a  typical  UAV  system  is  given  in  Figure  2,  including  a  single 
UAV,  a  GCS  with  two  operators,  and  an  image  analyst,  who  represents  either  a  unit 
commander  or  a  “customer”  of  the  UAV  payload  data. 

The  links  depict  communication  (T/R  =  Transmitter;  R/V  =  Receiver)  between  the 
corresponding  links.  For  the  purposes  of  this  report,  the  methods  used  to  implement 
these  communications  are  not  critical. 


5 


Human  Systems  Integration  and  Automation  Issues  in  SUAVs 


Figure  2.  Schematic  diagram  of  a  UAV  system. 
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PROBLEM  DEFINITION 

The  current  performance  of  SUAVs  has  room  for  improvement,  reflecting  a  technology 
that  is  still  under  development.  Some  of  the  challenges  that  must  be  faced  are  that 
SUAVs: 

•  are  manpower  intensive; 

•  have  a  large  “footprint”; 

•  need  to  improve  visual  data  display;  and 

•  have  a  high  loss  rate. 

Manpower  Intensive 

Current  Army  practice  is  to  assign  up  to  24  people  and  three  UAVs  per  UAV  system. 
Typically,  only  one  of  the  three  UAVs  is  airborne  at  a  given  time,  yielding  a  “worst  case” 
24:1  ratio  of  humans  to  UAVs.  Current  UAVs  are  controlled  from  the  GCS  with  two 
operators  and  often  two  additional  people — an  image  analyst  and  a  unit  leader.  Thus, 
the  “Human-to-UAV”  ratio  could  be  characterized  as  4:1,  or  3:1 ,  or,  best  case,  2:1 . 

The  Defense  Advanced  Research  Project  Agency  (DARPA)  has  attempted  to  push  the 
limits  on  human  control  of  automata,  including  UAVs.  The  objective  of  the  DARPA 
Mixed-Initiative  Control  of  Automa-teams  (MICA)  program  was  to  design  a  system  that 
would  enable  30  humans  to  control  300  entities,  or  a  ratio  of  1 :10.  That  would  represent 
somewhere  between  a  20-  to  240-fold  improvement  over  current  SUAV  manning 
requirements.  This  challenge  has  been  called  “inverting  the  control  ratio” 

(Johnson,  2003). 

Manpower  costs  are  the  largest  driver  of  system  life  cycle  costs.  Improving  the  ratio  of 
humans  to  UAVs  will  enable  major  reductions  in  the  life  cycle  cost  of  a  UAV  system. 
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SUAV  Footprint 

The  term  “footprint”  is  used  to  convey  both  physical  and  logistical  footprint.  Future 
concepts  of  Small  or  Micro-UAVs  envision  air  vehicles  small  enough  and  light  enough  to 
be  carried  in  a  backpack,  hand-launched,  and  managed  by  one  person.  The  current 
reality  is  that  a  manned  SUAV  system  includes  the  following  components: 

•  GCS,  housed  and  transported  in  a  HMMWV; 

•  Antennas,  generators,  and  communications  gear; 

•  Remote  Video  Terminal; 

•  24  people; 

•  3  UAVs  (4  for  Hunter);  and 

•  Consumable  spares. 

The  physical  and  logistical  footprint  is  large  and  unduly  cumbersome  to  achieve  the 
benefit  of  one  airborne  UAV.  The  challenge  is  to  develop  future  SUAV  systems  that  are 
smaller,  lighter,  more  agile,  and  require  less  logistic  support. 

Visual  Data  Display  and  Search  Effectiveness 

Time  for  Visual  Target  Detection.  Visual  search  is  a  demanding,  time-consuming  task 
that  requires  multilayered  analysis  of  human  perceptual  and  cognitive  systems 
(Neisser,  1967).  In  one  target  detection  experiment  (Itti,  Gold,  and  Koch,  2001),  the 
participants  were  instructed  to  detect  a  target  in  a  natural  scene  photograph;  a  task 
similar  to  UAV  search  task.  The  average  time  required  for  target  detection  was 
2.8  seconds.  When  UAV  operations  lead  to  a  high  rate  of  video  flow,  the  target  may  not 
remain  visible  on  the  monitor  for  2.8  seconds.  Operational  factors  that  contribute  to  this 
visual  search  problem  are  unstabilized  imagery,  narrow  field  of  view  (FOV), 
low  visibility,  communication  drop  outs,  and  high  rate  of  visual  flow  (high  speed  and/or 
low  altitude). 

Figure  3  is  a  representation  of  the  time  available  to  the  image  analyst  to  detect  a  target 
from  the  visual  scene  as  a  function  of  altitude  and  speed.  The  assumptions  underlying 
the  model  are: 
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•  The  UAV  is  flying  at  constant  speed  over  ground  (in  [km/hour])  and  height  (in  [ft]). 
Typical  air  speed  for  existing  SUAVs  is  in  the  80  to  100  km/hr  range. 

•  The  camera  is  stabilized. 

•  The  target  is  stationary. 

•  There  are  no  lags,  or  other  errors. 

•  The  FOV  (30°)  of  the  camera  corresponds  to  a  typical  value  found  in  existing 
SUAVs. 

Under  these  assumptions,  the  plots  in  Figure  3  depict  the  time  available  for  detection  by 
the  image  analyst.  Clearly,  low  speed  and  high  altitude  provide  more  time  for  the 
analyst.  If  the  2.8-second  limit  is  applied  to  this  theoretical  plot,  we  find  that  only 
altitudes  above  500  feet  are  likely  to  result  in  target  detection. 
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Figure  3.  Available  time  to  detect  a  target. 

Visual  Data  Display  Challenges.  The  demand  for  “Tactical”-size  UAV  services  in  Iraq 
and  Afghanistan  indicates  that  commanders  in  the  field  place  a  high  value  on  the  tactical 
data  provided  by  the  TUAVs.  However,  observation  of  SUAV  operations  at 
Camp  Roberts  and  discussions  with  operators  indicate  that  improved  imagery  and 
image  stabilization  are  needed  to  improve  the  ISR  capability  of  SUAVs. 
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Several  issues  were  observed  to  influence  ISR  effectiveness  during  SUAV  search 
operations  at  Camp  Roberts.  The  objective  of  one  set  of  exercises  was  to  detect  and 
classify  three  types  of  military  ground  vehicles  in  an  area  approximately  2x3  km.  The 
terrain  was  hilly  with  clusters  of  oak  trees  and  occasional  dirt  roads.  Two  types  of 
SUAVs  were  engaged  in  the  search,  with  a  surrogate  HALE  UAV,  the  Pelican,  operated 
by  the  Center  for  Interdisciplinary  Remotely  Piloted  Aircraft  Studies  (CIRPAS).  The 
three  Army  ground  vehicles  followed  a  script  for  periods  of  movement  followed  by  either 
a  stop  with  no  concealment  or  partial  concealment  under  a  tree.  One  of  the  SUAVs  had 
a  fixed  “lipstick”  type  of  camera  and  the  other  SUAV  had  a  larger,  pan/tilt/zoom  (PTZ) 
camera. 

Based  on  our  observations,  camera  type  had  a  large  influence  on  ISR  effectiveness. 

The  searches  conducted  with  a  fixed  camera  were  unproductive.  If  the  altitude  were 
high  enough  (approximately  2,000  ft  above  ground  level)  to  support  a  reasonable  field  of 
view,  then  the  size  (retinal  angle  subtended  by  the  displayed  target)  and  resolution  of  the 
target/object  images  was  insufficient  to  enable  detection  by  the  human  observer,  given 
the  “bounce”  of  the  imagery  caused  by  mild  turbulence.  If  the  SUAV  altitude  were  low 
enough,  then  the  field  of  view  (sweep  width)  was  small,  the  optical  flow  rate  was  high, 
and  the  time  available  for  detection  was  low.  The  image  analyst’s  task  was  difficult  at 
low  altitude  due  to  the  combination  of  the  high  rate  of  optical  flow  and  the  “bounce”  of 
the  imagery. 

The  imagery  bounce  was  considerably  more  than  a  small  vibration.  It  was  observed  to 
be  a  magnitude  of  nearly  one  video  frame,  oscillating  irregularly  at  a  frequency  of 
approximately  2-4  Hz.  For  example,  if  an  object  were  at  the  top  of  the  monitor  image,  it 
would  go  near,  or  off,  the  bottom  of  the  monitor  image  several  times  per  second.  A 
quantitative  analysis  of  this  effect  is  recommended,  leading  to  recommendations  for 
imagery  stabilization. 

Searches  conducted  with  a  stabilized  PTZ  camera  operated  by  a  MPO  were  far  superior 
to  the  fixed-camera  system.  Imagery  from  the  Navy’s  TERN  UAV,  based  on  a  PTZ 
camera,  but  nonoptimal  optics,  was  adequate  for  the  detection  task.  The  surrogate 
HALE  UAV  was  not  used  as  a  primary  search  sensor,  but  it  provided  imagery  of 
extraordinary  quality,  stability,  and  resolution. 
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In  the  opinion  of  this  observer,  the  fixed  “lipstick”  camera  was  insufficient  to  support 
ISR  operations.  It  was  extremely  difficult  to  detect  a  military  vehicle  parked  in  the  open 
at  a  known  crossroads  and  nearly  impossible  to  detect  a  vehicle  partially  obscured  under 
a  tree.  Perhaps  this  type  of  SUAV  search  system  could  provide  useful  imagery  under 
constrained  conditions,  such  as  very  smooth  air  and  open  areas  with  no  trees  or  other 
concealment.  In  the  Camp  Roberts  exercises,  systems  with  a  shock-mounted, 

PTZ  camera  provided  far  better  imagery  for  the  detection  task,  but  that  performance 
comes  with  a  weight  penalty.  Some  research  has  been  done  on  the  development  of  an 
intelligent,  semiautonomous  interface  for  stabilization  of  UAV  camera  images 
(Korteling  and  van  der  Borg,  1997).  The  relationship  between  image  quality,  probability 
of  detection,  and  the  weight  penalties  that  come  with  a  shock-mounted,  PTZ  camera 
need  to  be  quantified  to  support  future  SUAV  design  decisions. 

One  HSI  challenge  is  to  define  the  detection  system  requirements  by  working 
“backwards”  from  the  observer  to  the  sensor.  Existing  methods  for  characterizing 
sensor  resolution,  like  the  National  Imagery  Interpretability  Rating  Scales  (NIIRS)  or 
Johnson  criteria  (Leachtenaur  and  Driggers,  2001),  need  to  be  extended  to  characterize 
the  performance  of  trained  human  analysts  monitoring  a  streaming  video  image.  This 
analysis  is  needed  not  only  for  static  optical  resolution,  but  for  dynamic  issues  such  as 
optical  flow,  vibration,  and  image  bounce  caused  by  turbulence  and  the  aerodynamic 
response  of  the  air  vehicle. 

Additionally,  research  is  needed  to  determine  the  probability  of  detection  for  streaming 
video  imagery  compared  to  a  series  of  still  images.  The  frequency  and  duration  of  still 
images  are  variables  that  need  to  be  tested.  One  design  option  is  to  allow  the  operator 
(image  analyst)  to  select  the  image  iteration  rate.  Tests  are  needed  to  determine 
whether  operator  detection  performance  with  unstabilized  video  imagery  is  better  when 
viewing  a  series  of  static  frames.  Research  on  this  issue  should  be  done  with  both  static 
and  moving  targets.  It  is  possible  that  static  images  may  be  better  for  stationary  targets, 
while  streaming  video  could  have  an  advantage  for  detecting  moving  targets. 

Experts  in  visual  perception  should  analyze  the  engineering  characteristics  of  UAV  video 
displays  to  optimize  visual  performance  and  pattern  recognition.  Engineering  progress 
in  reducing  the  size  and  weight  of  UAVs  will  be  beneficial  only  if  the  video  product  is 


11 


Human  Systems  Integration  and  Automation  Issues  in  SUAVs 


useful,  i.e.,  stabilized  and  with  sufficient  resolution  and  contrast  for  the  analyst/observer 
to  achieve  acceptable  detection/classification  performance. 

UAV Loss  Rate 

The  loss  rate  for  UAVs  is  several  times  that  for  manned  aircraft.  The  reliability  and 
sustainability  of  UAVs  establishes  the  basis  for  their  affordability  and  their  mission 
availability  (DoD,  2002).  Accurate  data  on  UAV  reliability  are  difficult  to  obtain  and  that 
seems  particularly  true  for  SUAVs.  There  is  no  standardized  database  for  UAV  mishap 
reporting  and  there  are  a  large  number  of  UAV  systems  developed  by  a  large  number  of 
manufacturers  and  operated  by  different  branches  of  the  military.  A  definition  of  UAV 
reliability  comprising  four  metrics  has  been  proposed  (DoD,  2002): 


1 )  Mishap  Rate  is  the  number  of  accidents  occurring  per  1 00,000  flight  hours. 

2)  Mean  Time  Between  Failure  (MTBF)  is  the  ratio  of  hours  flown  to  the  number  of 
maintenance-related  cancellations  encountered  (expressed  in  hours). 

3)  Availability  is  the  number  of  times  a  given  aircraft  type  is  able  to  perform  its 
missions  compared  to  the  number  of  times  it  is  tasked  to  do  so,  expressed  as  a 
percentage  (describes  the  performance  of  a  system  while  on  standby). 

4)  Reliability  is  100  minus  the  percentage  of  times  a  launched  mission  is  either 
canceled  before  takeoff  or  aborted  during  flight  due  to  maintenance  issues, 
expressed  as  a  percentage  (describes  the  performance  of  a  system  while  in 
operation). 


Table  1.  Four  UAV  Reliability  Metrics  (derived  from  DoD,  2002). 


System 

Mishap  Rate* 
per  100,000  hrs. 

MTBF 

(hrs.) 

Availability 

Reliability 

Predator 

32 

44 

67% 

82% 

Pioneer 

334 

14 

76% 

86% 

Hunter 

55 

11 

98% 

82% 

*Class  A  mishaps,  resulting  in  the  loss  of  the  aircraft,  a  death,  or  more  than 
$1 ,000,000  in  damage;  for  two  model  series  (RQ-1A  &  -IB;  RQ-2A  &  -2B;  RQ-5 
pre-/post-1996). 


No  data  were  reported  for  small  UAVs,  given  that  the  Hunter  is  usually  classified  as  a 
“Tactical”  UAV.  The  poor  MTBF  results,  however,  are  consistent  with  our  observations 
at  the  Naval  Postgraduate  School  experiments  at  Camp  Roberts,  where  four  different 
types  of  UAVs  were  flown  on  multiple  sorties  on  multiple  days.  While  no  official  record 
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of  UAV  losses  is  available,  it  was  clear  that  mishaps  were  frequent,  on  the  order  of  one 
every  other  day.  Based  on  an  estimate  of  8-12  flight  hours  per  day,  that  equates  to 
approximately  one  mishap  per  16-24  flight  hours.  Discussions  with  one  group  of 
experienced,  military  SUAV  operators  at  Camp  Roberts  led  to  their  estimate  that,  for 
their  type  of  SUAV,  the  average  airframe  life  duration  is  approximately  20  flight  hours. 
This  figure  is  consistent  with  the  MTBF  data  reported  by  DoD  (2002)  in  Table  1 .  It  is 
also  consistent  with  the  report  that  NATO  lost  20  to  30  UAVs  during  the  78-day  Kosovo 
air  campaign  (Mouloua,  Gilson,  and  Hancock,  2003). 

In  the  series  of  experiments  coordinated  by  the  Naval  Postgraduate  School  at 
Camp  Roberts,  the  causes  and  contributing  factors  of  mishaps  often  were  uncertain.  In 
some  cases,  however,  the  cause  was  obvious,  such  as  running  out  of  fuel.  Most,  if  not 
all,  current  UAVs  have  no  fuel  gauge.  The  operators  estimate  the  flight  duration  and 
recover  the  UAV  according  to  that  schedule.  When  operational  requirements  place  a 
high  value  on  a  few  more  minutes  on  station,  the  fuel  reserve  may  disappear,  especially 
when  high  ambient  temperature  has  increased  the  fuel  burn  rate.  As  a  side  note,  the 
tarmac  temperature  at  Camp  Roberts  on  one  day  of  operations  was  146°  F,  probably 
similar  to  the  temperatures  experienced  in  Iraq. 

Other  SUAV  losses  observed  at  Camp  Roberts  were  due  to  delamination  of  the  wing 
(perhaps  from  long-term  storage  in  very  hot,  humid  conditions  in  the  southeastern  U.S.) 
or  to  an  infrequent,  but  irksome,  software  bug  that  shuts  off  the  engine. 

A  preliminary  analysis  of  48  UAV  mishaps  was  reported  by  the  DoD’s  Human-Systems 
Information  Analysis  Center  (HSIAC),  based  on  data  from  the  Air  Force  and  Army  Safety 
Centers  (Rogers,  Palmer,  Chitwood,  and  Hover,  2004).  This  analysis  included  10  years 
of  data  for  Class  A  and  Class  B  mishaps,  defined  as  follows: 

Class  A  Mishap:  Damage  costs  of  $1 ,000,000  or  more  and/or  destruction  of 
an  aircraft,  missile  or  spacecraft  and/or  fatality  or  permanent  total  disability  of 
personnel. 

Class  B  Mishap:  Damage  costs  of  between  $200,000  and  $1 ,000,000 
and/or  permanent  partial  disability  and/or  three  or  more  people  hospitalized 
as  inpatients. 
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Of  the  48  mishaps,  approximately  one-third  (15)  were  attributed  to  mechanical  failure 
and,  according  to  the  judgments  of  Rodgers  et  al.  (2004),  all  of  the  remaining  33 
mishaps  involved  some  form  of  human-systems  issues.  The  criteria  for  these  judgments 
are  unclear  and  unspecified  in  the  report. 

The  cost  of  the  48  mishaps  was  nearly  $14  million  for  the  Army  and  $177  million  for  the 
Air  Force,  with  the  overall  average  cost  per  mishap  close  to  $4  million.  A  breakout  by 
UAV  type  for  the  48  mishaps  is  as  follows:  Predator  32%,  Hunter  19%,  Shadow  17%, 
Global  Hawk  8%,  others  24%. 

These  data  do  not  provide  insight  into  SUAV  loss  rates  because  they  will  be  less  than 
the  “Class  B”  definition  of  $200,000.  While  cost,  fatalities,  and  injuries  are  the  traditional 
measures  of  “loss”  for  manned  aircraft  mishaps,  and  may  be  appropriate  for  large  UAVs, 
they  may  not  be  the  appropriate  criteria  for  SUAVs.  SUAV  mishaps  are  not  terribly 
expensive,  nor  are  they  fatal,  but  they  compromise  ISR  effectiveness  and  add  to  logistic 
burdens.  When  an  SUAV  is  engaged  in  a  reconnaissance  mission,  search  time  and 
coverage  are  lost  if  it  goes  down  because  it  takes  time  to  prepare  and  launch  another 
SUAV  to  replace  it.  During  that  time,  the  target  or  object  of  the  search/tracking  mission, 
such  as  military  vehicles  or  mobile  launchers,  may  have  moved  to  cover.  If  SUAV  loss 
rates  are  high,  a  given  unit  may  not  have  sufficient  spares  to  provide  SUAV  ISR 
coverage. 

The  HSIAC  preliminary  analysis  of  Rodgers,  et  al.  (2004)  suggests  that  two-thirds  of 
SUAV  mishaps  involve  human  systems  integration  issues.  This  estimate  is  consistent 
with  historical  estimates  of  “human  error”  as  a  causal  or  contributing  factor,  which 
typically  are  in  the  range  of  60%-80%  of  all  accidents  in  ground  transportation,  aviation, 
and  industry  (Perrow,  1999).  Given  the  current  human-in-the-loop  design  of  UAV  control 
systems,  we  have  no  reason  to  believe  that  SUAV  operations  will  be  different. 

A  study  of  the  role  of  human  and  organizational  factors  in  Army  UAV  accidents  was 
reported  by  Manning,  Rash,  LeDuc,  Koback,  and  McKeon  (2004).  The  authors  identified 
56  UAV  accidents  from  the  U.S.  Army  Safety  Center’s  database  during  the  period 
FY95-FY03.  The  Army  traditionally  identifies  three  basic  causes  of  accidents — human, 
materiel,  and  environmental  factors.  This  report  summarizes  categories  of 
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human-causal  factors  such  as  workload,  fatigue,  SA,  training,  crew  coordination,  and 
ergonomic  design.  Human  error  was  found  to  play  a  role  in  approximately  one-third 
(32%)  of  the  reported  accidents.  Two  methods  of  accident  analysis  were  employed  and 
both  identified  individual  unsafe  acts  or  failures  as  the  most  common  human-related 
causal  factor  in  UAV  accidents.  There  are  at  least  two  important  conclusions  from  the 
Manning  et  al.  (2004)  analysis: 

1)  human-causal  factors  play  a  role  in  a  substantial  proportion  (one-third)  of  UAV 
accidents;  and 

2)  the  Army’s  current  accident  reporting  method  does  not  support  accurate  capture 
of  human  error  accident  data. 

A  systematic  basis  for  tracking  SUAV  losses  is  needed.  Accurate  data  on  SUAV 
mishaps  are  needed  to  support  analyses  of  causal  and  contributing  factors  so  that 
improvements  in  reliability  can  be  achieved. 

There  are  two  challenges  here: 

1)  Improve  SUAV  reliability  (the  HSI  specialists  should  focus  on  the  subset  of 
mishaps  involving  human  operators  or  maintainers);  and 

2)  Develop  a  standardized  reporting  system  and  database  for  UAV  reliability 
(including  SUAVs)  across  all  services. 

Acquisition  methods  for  some  SUAVs  seem  to  have  circumvented  the  traditional 
requirements  for  reliability  analyses  and  Government  acceptance  testing,  which  may 
contribute  to  the  mishaps  attributable  to  hardware  and  software  problems. 
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HSI  ISSUES  FROM  NEAR-TERM  AND  LONG-TERM  PERSPECTIVES 

Throughout  the  history  of  UAV  development  there  has  been  little  or  no  attention  paid  to 
applying  “best  practices”  of  HSI  and  user-centered  design.  But,  UAV  programs  have 
matured  to  the  point  that  haphazard  user  interface  design  is  no  longer  acceptable  and 
should  not  be  allowed  to  contribute  to  high  mishap  rates  and  reduced  system 
effectiveness. 

The  following  categories  of  HSI  issues  apply  to  UAV  operations,  both  now  and  in  the 
future: 


•  Human  Roles,  Responsibilities  and  Level  of  Automation; 

•  Command  and  Control;  Concept  of  Operations; 

•  Manning,  Selection,  Training,  and  Fatigue; 

•  Difficult  Operational  Environments; 

•  Procedures  and  Job  Performance  Aids;  and 

•  Moving  Control  Platforms. 

How  can  HSI  improve  SUAV  operations?  One  way  to  address  this  question  is  to 
consider  two  different  perspectives:  near-term  improvements  that  require  no  new 
technologies  and  longer-term  improvements,  such  as  semiautomated  systems,  that  will 
require  time  and  research  and  development  investment. 


Near-Term  Perspective 

Human  Roles,  Responsibilities,  and  Level  of  Automation 

The  development  of  current  SUAV  systems  has  focused  on  the  fundamental  enabling 
technologies  such  as  aeronautical  design,  sensors,  and  communication  links  with  the 
air  vehicle.  The  simplest  SUAVs  are  essentially  hobbyist  model  planes  with  a  small 
camera  installed.  The  human  role  in  this  type  of  system  is  to  manually  operate  a 
remotely  piloted  vehicle  or  teleoperated  system. 

Morphew  (2003)  interviewed  a  number  of  TUAV  operators  (AVOs  and  MPOs  )  and  found 
that  the  most  difficult  tasks  were  considered  to  be: 
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•  Launch  and  landing; 

•  Identifying  and  reacting  in  emergencies;  and 

•  Maintaining  situational  awareness  (SA). 

Technologies  are  under  development  to  support  UAV  operators  in  accomplishing  these 
tasks.  Several  flight  management  systems  have  been  developed  and  are  currently  in 
use.  For  example,  Piccolo  Plus  by  Cloud  Cap  is  a  “user  programmable  autopilot  for 
UAVs.”  This  technology  provides  basic  automation  functions  such  as  altitude  hold  and 
waypoint  following.  These  automation  features  reduce  the  operator  workload  by  taking 
him/her  out  of  the  continuous  manual  control  loop.  They  also  change  the  role  of  the 
human  operator  from  direct  control  to  a  hybrid  of  direct  and  supervisory  control. 

Further  development  is  needed  to  provide  a  mission  management  system  that 
comprises  premission  planning  and  dynamic  replanning  during  a  mission.  A  mission 
management  system  for  SUAVs  might  contain  the  following  features: 

•  Preprogram,  reprogram,  and  manage  the  flight  profile; 

•  Automated  takeoff  and  landing; 

•  Preprogram,  reprogram,  and  manage  the  sensor  payload; 

•  Decision  aiding  for  emergencies  and  degraded  mode  operations;  and 

•  Supplemental  displays  to  support  SA. 

This  type  of  mission  management  capability  exists  in  U.S.  military  systems — no  new 
technology  is  necessary.  Worthwhile  objectives  of  a  mission  management  system  are  to 
reduce  the  mishap  rate,  shrink  the  24-person  SUAV  unit,  and  enable  simultaneous 
control  of  multiple  UAVs. 

The  capability  to  manage  and  revise  the  flight  profile  and  sensor  payload  is  essential 
because,  according  to  experienced  military  operators  from  Fort  Huachuca  and 
Patuxent  River,  the  majority  of  all  SUAV  missions  require  revision  of  the  mission  plan 
during  the  mission.  The  operator  must  be  afforded  the  capability  to  change  parameters 
quickly  and  easily  via  an  excellent  user  interface.  Reverting  to  manual  control  should  be 
an  available  option,  but  revising  the  flight  profile  should  be  possible  without  going 
manual. 
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In  the  future,  the  role  of  the  human  will  expand  to  include  control  of  multiple  SUAVs  and 
control  from  a  moving  vehicle  (ground  or  air).  These  capabilities  and  role  changes  must 
be  accompanied  by  changes  in  the  user  interfaces,  training,  and  operating  procedures. 
The  “integration”  of  HSI  is  a  central  concept  in  evolving  the  capability  of  SUAV  systems. 
A  challenge  for  DoD  acquisition  is  to  manage  this  evolution,  rather  than  to  acquire  new 
hardware/software,  then  subsequently  deal  with  the  inevitable  problems  that  accompany 
post  hoc  development  of  operating  concepts,  procedures,  training,  and  user  interface 
functionality. 

Command  and  Control  and  Operations 

SUAV  operators  need  to  know  where  the  sensor  is  pointing.  This  capability  is  not 
provided  in  most  current  SUAVs.  The  coordinates  of  the  center  of  the  sensor  field  of 
view  are  essential,  for  example,  when  an  image  analyst  requests  a  more  thorough 
investigation  of  a  possible  contact.  Having  Global  Positioning  System  (GPS)  data  on  the 
location  of  the  air  vehicle  is  not  sufficient.  The  coordinates  of  the  sensor  view  are 
necessary.  A  geographic  tactical  display  is  needed  to  show  a  historical  track  of  the 
sensor  coverage.  This  display  mode  would  make  gaps  in  coverage  immediately  evident 
to  the  MPO  and  image  analysts.  Ideally,  the  sensor  history  track  would  have  a  decay 
function  selectable  by  the  operator. 

System  response  time  and  the  temporal  aspect  of  team  communications.  During 
search  operations  at  Camp  Roberts,  time  lags  were  observed  between  an  image  analyst 
communicating  “possible  contact”  and  the  coordinated  response  from  the  AVO  and  the 
MPO.  This  problem  was  exacerbated  when  the  operators  did  not  have  information 
about  the  coordinates  of  the  possible  contact.  The  time  lags  can  be  characterized  as  an 
inside-  and  outside-loop  control  system.  The  AVO  and  MPO  have  inside-loop  control  of 
the  UAV  and  the  sensors.  If  they  are  using  autopilot  features,  then  they  must  reacquire 
manual  control  and  take  appropriate  action.  Time  is  required  for  the  image  analyst  to 
assess  the  video  data,  perceive,  decide,  and  communicate  the  possible  contact.  Then, 
time  is  required  for  the  operators  to  respond.  There  are  several  possible  solutions  to  this 
problem.  The  analyst,  whether  local  or  remote,  could  be  given  a  manual,  rather  than 
verbal,  response  option  that  would  signify  “possible  contact”  and  capture  the  relevant 
coordinates.  Another  approach  is  team  training.  The  search  team  can  practice  the 
specific  communication  and  performance  necessary  to  minimize  response  times. 
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ISR  search  pattern  analysis  and  codification  in  a  “search  template”  tool.  SUAV 
operators  currently  define  a  search  pattern  by  manually  entering  data  for  a  series  of 
waypoints.  This  process  is  time  consuming,  tedious,  and  may  not  reflect  analyses  of 
alternative  search  patterns.  The  field  of  Operations  Research  can  develop  search  and 
detection  models  that  will  estimate  the  probability  of  detection  for  various  search 
patterns.  A  software  tool  could  be  provided  to  SUAV  operators  that  would  support 
optimal  search  patterns  while  eliminating  the  requirement  to  enter  every  waypoint.  With 
the  appropriate  sensor  performance  data  loaded  into  the  software  tool,  operational 
variables  such  as  altitude,  sensor  type  and  FOV,  type  of  target,  type  of  terrain, 
temperature,  and  winds  could  be  included  in  a  “Search  Template”  software  tool.  The 
operator  would  enter  points  defining  the  bounds  of  the  search  area,  and  then  use  the 
Search  Template  tool  to  define  the  UAV  track  that  provides  optimal  coverage. 

Operations  analysis  leading  to  recommendations  for  system  improvements.  Most 
current  SUAV  systems  were  developed  and  fielded  without  the  benefit  of  systems 
engineering  and  analysis.  A  thorough  analysis  of  current  SUAV  operations,  procedures, 
and  training  is  recommended  to  determine  ways  to  improve  SUAV  system  effectiveness. 
A  composite  team  of  experts  in  aeronautics,  sensors,  command  and  control, 
communications,  software,  tactics  and  human  factors/HSI  would  be  ideally  suited  to 
accomplish  these  analyses. 

Manning,  Selection,  Training,  and  Fatigue 

UAVs  are  not  unmanned  systems.  People  operate  UAVs  and  how  the  people  are 
selected,  trained,  and  scheduled  contributes  to  system  effectiveness. 

The  manning  criteria  for  SUAV  operations  should  be  reviewed  periodically  as  the 
systems  evolve.  At  the  present  time,  operation  in  the  National  Air  Space  is  unlikely  for 
SUAVs  and  the  Army’s  current  96U  MOS  appears  to  be  sufficient  for  Army  SUAV 
operations.  As  the  technology  changes  and  the  operational  concepts  change,  both  the 
number  of  personnel  in  a  unit  and  the  entry  qualifications  will  need  to  be  reevaluated 
periodically. 

An  enlightening  study  would  be  to  determine  the  probability  of  detection  and  the  time  to 
detection  for  the  top  20%  and  the  bottom  20%  of  operator  skill  and  proficiency. 
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Observation  of  SUAV  operations  at  Camp  Roberts  indicated  substantial  differences  in 
operator  skill.  The  expert  operator  and  team  are  much  more  likely  to  be  successful.  The 
challenge  is  to  define  the  selection  criteria  and  continually  update  them  as  the  job  and 
the  pool  of  candidates  evolve. 

Strong  training  programs  produce  excellent  operators.  Our  unstructured  interviews  with 
operators  indicated  that  the  Army  has  a  strong  training  program  for  UAV  operators  at 
Fort  Huachuca.  The  architecture  of  UAV  systems  is  conducive  to  embedded  training  in 
GCS  design.  Mission  rehearsal  training  also  is  feasible,  given  that  a  database  of  the 
operating  terrain  and  expected  objects  of  interest  are  available  and  the  setup  tools  are 
provided  to  establish  meaningful  scenarios. 

Fatigue,  sleep,  and  circadian  rhythm  have  strong  effects  on  human  performance 
(Miller,  Nguyen,  Sanchez,  and  Miller,  2003).  Watch  schedules  for  UAV  operators  are 
particularly  important  when  engaged  in  day  and  night,  24-hour  missions.  Recent  data 
indicates  that  a  human  operating  on  no  sleep  in  a  24-hour  period  exhibits  signal 
detection  performance  equivalent  to  someone  with  a  0.08  blood  alcohol  level,  the  legal 
limit  in  most  states  (Doheny,  2004).  Manning  decisions  should  ensure  that  a  sufficient 
number  of  qualified  and  trained  personnel  is  available  to  man  a  watch  schedule. 

Pushing  a  crew  to  extend  work  schedules  will  result  in  performance  decrements,  no 
matter  how  dedicated  and  motivated  the  personnel  may  be. 

Difficult  Operational  Environments 

HSI  issues  in  difficult  environments  should  be  analyzed  to  determine  risk  areas  and  to 
identify  potential  solutions.  Night  operations  are  an  important  example.  Dark  adaptation 
phenomena  can  be  important  for  display  design.  The  possible  use  of  image 
intensification  or  night  vision  devices  by  UAV  operators  should  be  addressed.  Other 
environmental  issues  should  be  reviewed,  such  as  extreme  temperatures,  precipitation, 
and  poor  visibility  (sand,  dust,  fog).  Finally,  worst-case  scenarios  must  be  entertained, 
such  as  the  requirement  to  operate  in  chemical  and  biological  protective  gear  (MOPP  4). 
The  UAV  equipment  interfaces,  training,  and  procedures  must  be  capable  of  supporting 
effective  operations  in  all  environments. 
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Procedures  and  Job  Performance  Aids  (JPAs) 

System  effectiveness  will  be  improved  by  the  development  and  application  of  doctrine, 
standard  procedures,  teamwork  guidelines,  and  JPAs.  Development  of  these  policies, 
procedures,  and  job  aids  requires  input  from  expert  analysts  working  with  experienced 
operators.  HSI  experts  who  are  knowledgeable  in  job  design,  JPAs,  and  training  should 
be  included  in  this  process,  along  with  subject  matter  experts  (SMEs)  with  relevant 
SUAV  experience. 

The  Navy  has  conducted  an  aggressive  trial  program  in  JPA  applications  wherein  every 
job  onboard  the  USS  Preble  (DDG-88)  was  provided  with  a  JPA  in  the  form  of  a 
personal  digital  assistant  (PDA).  Initial  findings  were  strongly  positive,  indicating  that  the 
JPAs  reduced  training  time  and  supported  job  performance  improvements  (Booz  Allen 
Hamiliton,  2002).  The  same  benefits  can  be  enjoyed  by  the  UAV  community  by 
deploying  JPAs. 

Moving  Control  Platforms 

Controlling  UAVs  from  a  moving  platform  presents  system  design  challenges,  whether 
the  controller  is  in  the  air,  on  the  ground,  or  aboard  ship.  Human  controllers  will  be 
confronted  with  the  requirement  to  maintain  spatial  orientation  and  situational  awareness 
(SA)  of  the  controlled  UAV(s),  while  simultaneously  sensing  and  understanding  the 
dynamics  of  their  own  vehicle.  For  example,  De  Vries  and  Jansen  (2002)  found  that 
display  of  own-vehicle  motion  reduced  the  operators’  ability  to  perceive  the  direction  and 
flight  path  of  a  controlled  UAV. 

The  following  key  issues  should  be  addressed  to  evolve  the  capability  to  manage  one  or 
more  unmanned  vehicles  from  a  moving  vehicle: 

•  How  should  spatial  information  about  own-vehicle  and  controlled  vehicle(s)  be 
displayed  to  the  UAV  operators? 

•  Can  individual  differences  in  spatial  orientation  and  mental  rotation  be  used  as 
criteria  for  selecting  and  assigning  UAV  operators? 

•  What  types  of  training  and  simulation  systems  are  necessary  to  develop  the 
necessary  skills  to  manage  complex  multivehicle  dynamics? 
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Dowell,  Shively,  and  Casey  (2003)  found  that  a  heading  tape  display  is  more  effective 
than  a  compass  rose  display  for  supporting  UAV  operator  performance.  These  findings 
have  practical  utility,  but  in  the  future,  this  area  of  research  will  need  to  be  extended  to 
determine  how  a  moving  operator  (in  a  HMMVW  or  an  AH-64  Apache,  for  example)  can 
best  retain  spatial  orientation  and  SA  with  dynamic  displays  of  multiple  vehicle  dynamics 
(Durbin,  Havir,  Kennedy,  and  Schiller,  2003). 

In  addition  to  the  issue  of  navigation/spatial  orientation,  moving  vehicles  present  three 
other  HSI  challenges: 

1)  motion  sickness; 

2)  biodynamic  interference  with  manual  control;  and 

3)  head-mounted  display  (HMD)  bounce. 

Motion  sickness  occurs  in  these  situations  due  to  a  combination  of  actual  motion  plus 
“cybersickness”  (McCauley  and  Sharkey,  1992;  Morphew,  2003).  The  visual  cues  to 
motion,  perceived  by  information  displayed  on  the  HMD,  differs  from  the  motion 
perceived  by  the  inner  ear,  setting  up  a  “cue  conflict”  situation  (Reason  and  Brand, 
1975). 

Biodynamic  interference  is  simply  the  feed-through  to  manual  control  interfaces  from  the 
shock  and  vibration  of  rough  terrain,  transmitted  through  the  vehicle  suspension  to  the 
operator’s  body,  arms,  and  hands.  This  can  be  a  significant  problem,  depending  on  the 
vehicle,  the  speed,  and  the  terrain. 

HMDs  tend  to  resonate  at  certain  frequencies  of  vertical  axis  vibration,  which  are  typical 
in  ground  vehicles  (Sharkey,  McCauley,  Schwirzke,  Casper,  and  Hennessy,  1995).  As 
the  HMD  “bounces”  relative  to  the  face  and  eyes,  degraded  visual  performance  will 
occur. 

These  HSI  challenges  in  operating  UAVs  from  moving  platforms  will  depend  greatly  on 
type  of  vehicle,  the  terrain,  and  vehicle  speed.  For  these  reasons,  HMDs  may  not  be 
suitable  display  devices  for  UAV  controllers,  especially  in  moving  control  platforms. 
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LONGER-TERM  PERSPECTIVE 

Despite  decades  of  research  and  development  on  UAVs,  this  technology  is  still  primitive, 
with  the  possible  exception  of  the  more  advanced  and  expensive  HALE  UAVs.  Current 
SUAV  technology  continues  to  rely  heavily  on  the  human  controller  acting  as  a  remote 
controller.  Given  this  legacy  design  concept,  the  human  controller  is  not  capable  of 
directly  controlling  multiple  UAVs  effectively,  especially  in  challenging  conditions. 
Breakthroughs  in  UAV  system  capability  must  come  from  pushing  into  new  conceptual 
areas  such  as  intelligent,  semiautonomous  agents  of  varying  size  and  capability 
operating  as  a  team  in  response  to  “plays”  called  by  the  human  supervisory  controller. 

Automation 

Advances  in  technology  are  proceeding  rapidly  in  many  areas  that  are  relevant  to 
unmanned  systems  such  as  robotics,  semiautonomous  systems,  biomimetics,  and 
agent-based  systems.  A  brief  overview  of  selected  technology  topics  is  given  in 
Appendix  A.  The  term  “autonomy”  can  be  applied  to  all  of  these  technologies  because 
they  contribute  to  a  shift  away  from  the  human-in-the-loop  control  that  characterizes 
SUAV  control  systems. 

Automation  is  likely  to  play  a  large  role  in  future  UAV  systems,  but  should  not  be  viewed 
as  a  panacea.  The  introduction  of  highly  automated  systems  can  cause  serious 
difficulties  such  as  mode  confusion,  “automation  surprise,”  distrust,  complacency,  and 
the  “out-of-the-loop”  operator  performance  problem  (Endsley  and  Kiris,  1995; 

Sarter,  Woods,  and  Billings,  1997;  Sheridan,  1992;  Weiner,  1988). 

“Clumsy  automation”  is  the  term  used  to  describe  automation  that  makes  easy  tasks 
easier  and  difficult  tasks  more  difficult  (Weiner,  1988).  This  is  the  opposite  of  the 
intended  objective  of  automation,  which  is  to  reduce  peak  workload  (i.e.,  reduce  the 
difficulty  of  difficult  tasks).  Woods  (1993)  summarized  studies  on  automation  design  this 
way:  “New  technology  introduced  for  putative  benefits  in  fact  introduced  new  demands 
and  complexities  into  already  highly  demanding  fields  of  practice.” 

Many  automation  functions  could  be  applied  to  SUAVs,  ranging  from  improved 
“autopilot”  functions  to  autonomous,  collaborative  swarms.  The  former  functions  were 
operational  50  years  ago;  the  latter  are  not  yet  operational.  But,  ignoring  the  technical 
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feasibility  for  the  moment,  what  design  strategy  should  be  adopted  for  the  sequential 
introduction  of  automation  features  into  a  UAV  system? 

Human-in-the-loop  simulation  is  an  excellent  tool  for  addressing  HSI  issues  prior  to  final 
design  of  the  technological  subsystems.  The  manned-simulation  approach  enables  the 
tradeoffs  to  be  evaluated  between  the  technical  requirements,  including  automation 
features,  and  the  HSI  issues  such  as  operator  roles,  tasking,  manning,  workload,  user 
interfaces,  the  probability  of  operator  error,  and  training  requirements. 

Allocation  of  System  Functions  to  Automation  or  Humans 

As  automation  technology  evolves,  how  much  control  should  humans  relinquish?  What 
role  should  the  human  operator  play  in  UAV  systems?  The  answers  to  these  questions 
also  may  evolve,  but  these  HSI  and  systems  engineering  questions  need  to  be 
answered  prior  to  the  application  of  automation  “solutions.” 

What  functions  are  good  candidates  for  automation?  Frequently,  automation  is 
advocated  for  jobs  that  are  “dirty,  dull,  or  dangerous.”  Although  this  saying  has  good 
alliteration,  it  is  neither  an  exhaustive  list  nor  a  valid  set  of  criteria.  Systems  engineers 
and  HSI  specialists  have  the  opportunity  to  determine  in  advance  the  control  allocation 
architecture.  A  process  analogous  to  triage  may  be  warranted,  where: 

1 )  certain  functions  may  be  reserved  for  human  operators; 

2)  other  functions  are  shared,  mixed,  or  adaptively  allocated  to  the  best  available 
resource,  either  human  or  automated;  and 

3)  some  functions  may  be  routinely  allocated  to  the  automated  subsystems. 

Through  the  use  of  simulation,  the  challenge  of  designing  function  allocation  can  be 
addressed  early  in  system  design,  prior  to  engineering  or  technological  determinations 
about  how  ihe  automation  features  might  be  implemented. 

Table  2  gives  a  first  approximation  of  separating  top-level  (human  control),  mid-level 
(overlapped,  shared,  or  adaptive  control),  and  lower-level  tasks  (prime  candidates  for 
automation).  The  “top-level”  list  is  intended  to  endure,  no  matter  how  sophisticated  the 
technology  may  become  in  the  future.  The  mid-level  tasks  are  currently  accomplished 
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by  humans,  but  could  be  allocated,  at  least  some  of  the  time  (adaptively),  to  intelligent 
automation  in  the  future.  The  lower-level  tasks  also  are  frequently  accomplished  by 
humans,  especially  during  takeoff  and  landing,  but  current  technology  is  capable  of 
performing  these  functions. 


Table  2.  Example  of  Function/Task  Allocation  for  SUAV  Systems. 


Top-Level  Human  Tasks 

Mid-Level  Shared  Tasks 

Lower-Level  Automated  Tasks 

1.  Define  Mission  Objectives 

1.  Airspace  Deconfliction; 
“See  and  Avoid” 

1.  Vehicle  Control  (autopilot 
functions) 

2.  Specify/ Assign  Team  and 
Assets 

2.  Terrain  Avoidance 

2.  Counter  Environmental 
Disturbances 

3.  Specify  Location  (region, 
area) 

3.  Implement  Tactics 

3.  Maintain  Sensor  Lock-on 

4.  Specify  Time  Period 

4.  Navigate 

4.  Collision  Avoidance 

Processing 

5.  Determine  Tactics 

5.  Avoid  or  Investigate 
Possible  Threats 

5.  Store  or  Transmit  Data 

6.  Determine  Contingency 
Plans 

6.  Teamwork  Dynamics 

7.  Initiate  Operation 

7.  Sensor/Payload  Control 

8.  Modify  Mission  Objectives 
and  Tactics 

8.  Takeoff  and  Landing 

9.  Launch  Weapons 

10.  Terminate/Recall  Mission 

Sheridan  (1992;  2002)  has  provided  excellent  insights  into  automation,  supervisory 
control,  and  system  design.  As  mentioned  earlier,  a  common  error  is  to  automate 
whatever  functions  are  easy  to  automate  and  let  the  human  do  the  rest.  As  noted  by 
Sheridan  (2002): 

From  one  perspective  this  dignifies  the  human  contribution.  From 
another  it  may  lead  to  a  hodgepodge  of  partial  automation,  making  the 
remaining  human  tasks  less  coherent  and  more  complex  than  they  need 
be,  and  resulting  in  overall  degradation  of  system  performance. 

Advancements  in  the  human-system  engineering  processes  for  developing  automation 
functions  in  manned  military  aircraft  are  relevant  to  UAV  systems  (Taylor,  Bonner, 
Dickson,  Howells,  Miller,  Milton,  Pleydell-Pearce,  Shadbolt,  Tennison,  and  Whitecross, 
2002).  The  authors  advocate  adaptive  automation  and  decision  aiding  based  on 
cognitive  systems  engineering  principles.  Communication  of  intent  is  important 
because,  in  a  hierarchical  tasking  system,  the  intent  of  the  commander  flows  down 
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through  the  task  hierarchy.  The  concepts  and  methods  suggested  by  Taylor  et  al. 
(2002)  were  integrated  in  the  Cognitive  Cockpit  (COGPIT)  program  of  the 
United  Kingdom  Defence  Evaluation  Research  Agency  (DERA).  These  concepts  are 
applicable  not  only  to  aircraft  interfaces,  but  to  UAV  systems,  both  for  user  interface 
design  of  the  GCA  and  for  automation  system  design. 


One  approach  to  defining  types  and  levels  of  automation  has  been  suggested  by 
Parasuraman,  Sheridan,  and  Wickens  (2000).  This  model  or  framework  features  the 
selection  of  a  level  of  automation  for  each  of  four  stages  of  human  information 
processing,  as  shown  in  Figure  4. 

Figure  4.  Levels  of  automation  for  each  of  four  stages  of  human  information  processing 

(from  Parasuraman  et  al.,  2000). 
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According  to  this  model,  four  stages  of  human  information  processing  can  be  applied 
both  to  human  information  processing  and  to  automated  functionality.  The  design  of  any 
specific  system  can  apply  automation  at  a  selected  level  in  each  of  the  four  stages.  In 
the  example  given  in  Figure  4,  System  A  has  less  automation  than  System  B  and  has  its 
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highest  level  of  automation  in  the  “Information  Acquisition”  stage.  A  medium  to  high 
level  of  automation  in  this  first  stage  would  imply  support  for  sensor  management  and 
perhaps  some  degree  of  highlighting,  preprocessing,  or  preliminary  object 
detection/recognition  aiding. 

Parasuraman  et  al.  (2000)  emphasize  the  importance  of  testing  and  evaluating 
preliminary  choices  of  automation  functionality.  Iterative  testing  establishes  the  best 
automation  levels  for  supporting  human  operator  performance.  A  secondary  testing 
criterion  would  be  the  reliability  of  the  automation  functionality.  Reliability  of  automation 
is  important  for  achieving  the  trust  of  the  human  operator. 


There  have  been  several  attempts  to  define  stages  or  levels  of  automation. 
Sheridan  (1992)  provided  a  definition  often  levels  of  automation  (see  Table  3). 


Table  3.  Sheridan’s  “Scale  of  Degrees  of  Automation”  (from  Sheridan,  1992). 


1. 

The  computer  offers  no  assistance;  human  must  do  it  all. 

2. 

The  computer  offers  a  complete  set  of  action  alternatives,  and 

3. 

Narrows  the  selection  down  to  a  few  alternatives,  or 

4. 

Suggests  an  alternative,  and 

5. 

Executes  that  suggestion  if  the  human  approves,  or 

6. 

Allows  the  human  a  restricted  time  to  veto  before  automatic  execution,  or 

7. 

Executes  automatically,  then  informs  the  human,  or 

8. 

Informs  human  after  execution  only  if  asked,  or 

9. 

Informs  human  after  execution  if  it,  the  computer,  decides  to. 

10. 

The  computer  decides  everything  and  acts  autonomously,  ignoring  the  human. 

Some  of  the  HSI  challenges  in  developing  effective  automation  systems  in  UAVs  are  to 
develop: 


1 )  criteria  for  the  application  of  automation  features; 

2)  a  consistent  and  unambiguous  way  to  inform  the  user  about  the  status  of  modes 
and  automation  features;  and 

3)  a  design  that  empowers  the  human  user  to  reacquire  control  of  any  automated 
variable  upon  request. 
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HSI  Issues  in  Automation 

Distrust  and  Complacency.  When  automation  is  introduced,  the  job  of  human  controllers 
usually  is  shifted  from  direct  control  to  supervisory  control  (Sheridan,  1992;  2002).  A 
common  reaction  to  this  shift  in  roles  and  responsibilities  is  for  the  human  controllers  to 
distrust  the  automation  features.  This  problem  is  exacerbated  when  early  versions  of  an 
automated  system  are  perceived  to  be  unreliable,  which  can  be  expected  when  reliability 
is  less  than  95%. 

In  a  recent  review  of  trust  in  automation,  Lee  and  See  (2004)  defined  trust  as  “the 
attitude  that  an  agent  will  help  achieve  an  individual’s  goals  in  a  situation  characterized 
by  uncertainty  and  vulnerability.”  After  automation  has  been  integrated  into  ongoing 
operations  the  problem  of  “distrust”  is  more  likely  to  be  replaced  by  the  problem  of 
overtrust  or  complacency.  The  users  (human  operators,  supervisory  controllers)  trust 
the  automated  system  too  much,  leading  to  complacency  in  their  role  as  “supervisory 
controller.” 

Lee  and  See  (2004)  use  the  terms  “appropriate  trust”  or  “calibrated  trust”  because 
humans  could  err  in  either  direction — distrust  a  reliable  automation  feature  or  overtrust 
an  unreliable  automation  feature.  Distrust  leads  to  disuse  (the  capabilities  of  automation 
are  rejected),  whereas  overtrust  leads  to  complacency  and  misuse  (violation  of 
assumptions  or  inappropriate  reliance  on  automation). 

Design  and  training  approaches  are  suggested  by  Lee  and  See  (2004)  to  achieve 
appropriate  trust  in  automation,  including: 

•  Design  for  appropriate  trust,  not  greater  trust; 

•  Show  the  past  performance  of  the  automation; 

•  Show  the  process  and  algorithms  by  revealing  sensible  intermediate  results; 

•  Show  the  purpose  and  design  of  the  automation  in  a  way  that  relates  to  the 
user’s  goals;  and 

•  Train  operators  about  the  expected  reliability  and  intended  use. 
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Case  Example.  A  case  example  of  overtrust  or  complacency  was  observed  at 
Camp  Roberts  as  a  UAV  was  returning  to  the  landing  strip  under  automated  control  of 
altitude  and  heading.  Because  no  direct,  manual  control  was  being  exercised  and  no 
search  process  was  ongoing,  the  operator(s)  were  loosely  monitoring  the  video  image 
on  a  small  back-up  display.  The  unscheduled  descent  was  not  noticed  until  it  was  too 
late.  Upon  impact,  the  video  imagery  flow  became  static,  leading  one  operator  to 
exclaim,  “I  think  we  may  have  crashed!”  The  three  other  team  members  met  this 
statement  with  incredulity,  until  they  observed  the  static  video  close-up  of  the  ground. 

An  intermittent  software  error  was  thought  to  have  been  the  likely  source  of  the  problem, 
extending  landing  gear  and  cutting  power.  An  alert  operator,  monitoring  the  situation 
closely,  may  have  had  time  to  reacquire  manual  control  prior  to  impact.  Overtrust  in  the 
altitude-hold  automation  feature  may  have  contributed  to  the  failure  to  detect  the 
unintended  descent. 

Human  Error  in  Automated  Systems.  Reduction  of  human  error  often  is  given  as  a  basis 
for  the  decision  to  automate  system  functions.  If  the  human  is  removed  from  the  control 
loop,  he  cannot  make  an  error.  Or  can  he?  Early  use  of  automation  in  the  commercial 
airline  industry  provides  lessons  that  can  be  applied  for  UAV  control.  Because  a 
majority  of  airline  crashes  were  attributed  to  “human  error,”  many  technologists  in  the 
1970s  believed  that  automation  was  the  best  way  to  reduce  human  error.  Wiener  and 
Curry  (1980)  analyzed  cockpit  automation  as  the  “glass  cockpit”  was  introduced  into 
commercial  aviation  in  the  1970s.  Their  analysis  did  not  support  the  assumption  that 
automation  would  decrease  human  error.  Rather,  the  authors  suggested  that 
automation  changes  the  types  of  errors  and,  in  fact,  may  create  opportunities  for  new 
types  of  human-system  errors. 

Human  error  is  complex  (Reason,  1990).  One  reason  to  adopt  a  systems  engineering 
and  HSI  perspective  on  the  development  of  automated  systems  is  to  identify  in  advance 
various  forms  of  human  error  and  design  the  system  to  prevent  them  or  limit  their 
consequences. 

Mode  Confusion.  The  introduction  of  the  “glass  cockpit”  to  commercial  aviation  in  the 
1970s  required  a  transition  period  during  which  flight  crews  had  to  learn  new  controls, 
displays,  new  roles  (supervisory  control),  and  a  proliferation  of  computer-controlled 
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automation  features.  One  common  problem  with  the  new  feature-rich  automation 
systems  was  mode  confusion,  sometimes  resulting  in  “automation  surprise”  (Sarter  and 
Woods,  1995;  Sarter,  Woods,  and  Billings,  1997). 

This  problem  of  mode  confusion  and  automation  surprise  occurs  for  one  of  two  reasons: 

1 )  the  effect  of  a  specific  operator  (pilot)  input  is  mode-dependent;  or 

2)  the  operator  is  surprised  at  the  behavior  of  the  automated  system  because 
he/she  made  an  incorrect  assumption  (or  had  an  inadequate  mental  model) 
about  the  current  mode  of  operation. 

The  likelihood  of  such  an  error  is  more  apparent  when  one  understands  that  one  type  of 
commercial  airline  cockpit  has  1 1  auto  thrust  modes,  17  vertical  flight  path  modes,  and 
10  lateral  flight  path  modes.  This  type  of  feature  proliferation  and  complexity  sets  the 
stage  for  human  operators  to  suffer  mode  confusion,  to  enter  erroneous  data,  or 
otherwise  interact  with  the  automated  system  in  a  way  that  does  not  achieve  the 
intended  result. 

Mode  confusion  is  not  a  problem  with  current  UAVs,  but  will  be  arriving  soon,  as  more 
complex  automated  systems  are  introduced.  If  system  designers  focus  on  HSI  and  on 
applying  the  “lessons  learned”  from  the  introduction  of  automation  in  commercial  aviation 
over  20  years  ago,  the  result  will  be  a  more  graceful  introduction  of  automated  features 
into  UAV  systems. 

Integrating  human  controllers  into  automated  systems  is  a  critical  research  goal  for  UAV 
systems.  The  fundamental  engineering  technologies  that  enable  UAV  functionality,  such 
as  aeronautical  systems,  propulsion  systems,  sensors,  and  communications  links,  need 
continued  development.  But,  in  addition  to  those  fundamental  engineering  technologies, 
major  improvements  in  SUAV  effectiveness  and  reductions  in  life  cycle  cost  can  be 
obtained  from  human-centered  design  of  automation  features. 
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RESEARCH  RECOMMENDATIONS 

Throughout  this  report,  opportunities  for  research  and  development  have  been 
suggested  directly  or  implied.  The  following  list  is  a  compilation  of  research 
recommendations  mentioned  in  this  report,  sorted  by  major  categories: 

Automation 

1.  Determine  the  automation  features,  selection  criteria,  and  training  requirements 
that  would  be  needed  to  support  the  control  of  multiple  SUAVs  by  a  single 
operator. 

2.  Perform  research  on  defining  the  roles,  responsibilities,  and  tasks  for  human 
operators  as  automation  technology  evolves. 

3.  Develop  UAV  mission  management  (planner/replanner)  systems  to  reduce 
operator  workload. 

4.  Develop  a  search-aid  software  tool  to  assist  UAV  operators  to  select  and 
implement  good  search  tactics  for  given  situations. 

5.  Perform  research  on  adaptive  automation  systems,  mixed-initiative  systems,  and 
collaborative  control  to  determine  successful  strategies  and  designs  for 
semiautomated  UAV  systems. 

6.  Perform  research  to  establish  guidelines  for  avoiding  mode  confusion  and 
automation  surprise  in  semiautomated  UAV  systems. 

Manpower,  Personnel,  and  Training 

7.  Conduct  research  and  analysis  on  how  to  improve  the  ratio  of  humans  to  SUAVs 
without  incurring  a  reduction  in  system  effectiveness. 

8.  Reduce  the  manpower  requirements  of  a  SUAV  unit  by  implementing  HSI  design 
for  maintenance  and  flight  operations  support. 

9.  Develop  training  materials  for  the  instruction  and  qualification  of  image  analysts, 
including  mission  rehearsal  capability  to  detect,  classify,  and  identify  objects  and 
entities  relevant  to  upcoming  mission. 

10.  Develop  embedded  training  systems  to  promote  skill  acquisition  and 
maintenance  using  the  GCS  as  the  training  simulator. 

11.  Develop  personnel  selection  criteria  based  on  quantitative  measures  of  UAV 
system  performance  (e.g.,  probability  of  detection;  time  to  detection;  probability 
of  false  alarm;  spatial  orientation;  vehicle  control;  sensor  control). 
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Mishaps  and  Mishap  Reporting 

12.  Develop  a  standardized,  DoD-wide  reporting  system  for  UAV  mishaps. 

13.  Reduce  SUAV  mishap  and  loss  rates  by  analysis  of  losses  where  operator  errors 
are  cited  as  a  causal  or  contributing  factor. 

HSI  and  Systems  Analysis 

14.  Execute  a  thorough  HSI  and  systems  engineering  analysis  of  current  TUAV  and 
SUAV  operations  to  determine  major  contributors  to  success  (and  failure)  of 
current  systems  (aeronautics,  propulsion,  sensors,  communications,  control 
systems,  user  interfaces,  tactics,  and  utilization  of  data  in  the  larger  command 
and  control  environment). 

15.  Ensure  contribution  of  HSI  to  the  ongoing  process  of  the  design  and 
development  of  a  common  console  for  UAV  control. 

Policy  and  Procedures 

16.  Prepare  for  24/7  operations  by  developing  guidelines  for  managing  sleep  and 
fatigue  through  chronohygiene  and  watch-standing  procedures. 

17.  Design  and  train  for  difficult  environmental  conditions — night,  poor  visibility,  and 
chemical  defense  gear. 

Human  Factors  Engineering  of  System  or  Subsystem  Design 

18.  Define  UAV  detection  hardware  and  software  system  requirements  by 
establishing  criteria  (time,  contrast,  resolution,  visual  angle,  motion,  and  other 
relevant  factors)  for  human  detection  and  identification  of  mission-relevant 
objects. 

19.  Perform  research  to  determine  the  best  practices  for  using  static  and  dynamic 
imagery  to  support  human  target  detection. 

20.  Develop  hardware  and  software  to  stabilize  the  visual  imagery. 

21 .  Implement  the  technology  to  monitor  the  coordinates  of  a  camera  or  other  sensor 
aboard  SUAVs. 

22.  Develop  Job  Performance  Aids  (JPAs)  and  decision  aids  for  UAV  operators 

23.  Develop  design  guidelines  for  displays  and  other  system  features  that  will  enable 
UAV  operators  to  perform  their  tasks  in  a  moving  vehicle  (air,  ground,  or  surface) 
while  maintaining  spatial  orientation  and  situation  awareness. 
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24.  Develop  simulation  capabilities  to  enable  analysis  of  UAV  effectiveness  as  a 
function  of  new  technologies,  automation,  procedures,  displays,  and  other 
system  features. 

Research  Recommendations  from  Appendix  A 

25.  Perform  research  on  intelligent  agent  systems  and  multiagent  systems  as  a  basis 
for  improving  SUAV  effectiveness  while  reducing  the  UAV  footprint  and  operator 
workload. 

26.  Perform  research  on  levels  of  autonomy  and  semiautonomous  systems  with 
application  to  SUAV  systems. 

27.  Perform  research  on  swarms  and  stigmergy  to  determine  how  to  apply  them  to 
automation  in  SUAV  systems  and  micro  UAV  systems  with  continued  focus  on 
the  human  operator  role  as  supervisory  controller. 

28.  Determine  the  interchange  and  mutually  beneficial  overlap  between 
Human  Systems  Integration  and  Human  Robot  Interfaces  with  reference  to 
SUAV  applications. 

29.  Perform  research  on  how  hierarchical  control  architectures  like  Playbook  might 
be  implemented  in  SUAV  systems  with  one  human  operator  controlling  multiple 
SUAVs. 
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APPENDIX  A:  AUTONOMY  TECHNOLOGY 

This  overview  is  intended  to  identify  some  of  the  research  issues  that  are  being 
addressed  in  autonomy  and  related  disciplines  that  are  relevant  to  future  UAV  design 
and  operations.  It  is  not  a  comprehensive  review  of  the  scientific  and  technical  literature 
in  these  fields. 

Robotics,  Intelligent  Agents,  and  Multiagent  Systems 

UAV  development  can  be  considered  a  subfield  of  robotics  and,  thus,  advances  in 
robotics  are  important  to  UAV  design  and  performance.  Research  and  development 
programs  in  robotics  are  ongoing  at  leading  universities  in  the  U.S.,  Europe,  and  Asia. 

Robots  (including  UAVs)  face  many  of  the  same  challenges  that  human  perceptual 
systems  perform  so  well,  such  as  visual-inertial  stabilization  and  the  analysis  of  whether 
you  are  moving  or  the  environment  is  moving  (Panerai,  Metta,  and  Sandini,  2000). 

These  intersensory  processes  and  feedback  loops  provide  the  foundation  for  exploiting 
maximum  visual  acuity  (gaze  stability)  and  for  discriminating  between  self-motion  and 
motion  of  the  visual  scene.  Compensation  for  motion  of  either  sort  is  necessary  to 
maintain  image  stability  on  the  retina  (or  optic  sensors).  As  mentioned  earlier  in  this 
report,  the  lack  of  this  capability  has  been  observed  to  be  a  problem  for  small  UAVs 
without  image-stabilization  systems.  Image  stability  for  some  minimal  duration  is  a 
fundamental  requirement  in  an  optical  detection  system,  whether  a  human  analyst  or  an 
automated  agent  is  processing  the  image. 

A  demonstration  of  autonomous  robotic  capability  called  ’’The  Grand  Challenge”  was 
sponsored  by  the  Defense  Advanced  Research  Project  Agency  (DARPA)  in  March  2004. 
The  top  15  autonomous  ground  vehicles  participated  in  a  challenge  to  win  a  $1  million 
prize  by  navigating  140  miles  through  the  Mojave  Desert  from  Barstow,  California  to 
Primm,  Nevada.  The  entrants  included  teams  like  Cal  Tech,  Berkeley,  and 
Carnegie-Mellon  using  significantly  customized  vehicles  built  on  the  chassis  of 
Chevy  Tahoes,  HMMWVs,  and  other  off-road  vehicles.  Not  only  did  none  of  the  robots 
make  it  to  the  destination,  only  two  of  the  teams  made  it  as  far  as  seven  miles.  Most  of 
the  entries  failed  to  traverse  more  than  a  few  hundred  yards.  According  to  one  news 
item,  “it  was  a  pretty  humbling  display  for  unmanned  robot  vehicle  technology.”  The 
results  underline  DARPA’s  prescient  selection  of  the  event  name,  “The  Grand 
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Challenge.”  Obviously,  overcoming  difficult  environmental  obstacles  is  truly  a  grand 
challenge  for  autonomous  off-road  ground  systems  in  2004. 

The  results  of  the  2004  DARPA  Grand  Challenge  indicate  that  frequent  “reality  checks” 
(test  and  evaluation  under  operational  conditions)  are  necessary  as  autonomous 
technologies  are  developed  and  before  the  lives  of  U.S.  military  personnel  are 
dependent  on  their  success. 

Intelligent  agents  have  the  common  requirement  that  they  must  be  goal  directed.  An 
agent  must  exhibit  purposeful  behavior  and  have  an  explicit  representation  of  their  goal 
(Huber,  1999).  Research  on  multiagent  systems  (MAS)  has  commonly  led  to  hierarchies 
of  autonomy  with  various  decision-making  procedures  employed  by  the  agents.  MAS 
prototypes  have  demonstrated  capability  in  simulated  adversarial  games  such  as  robot 
soccer.  Stone  (1998)  applied  machine  learning  techniques  to  MAS  and  demonstrated 
that  a  group  of  independent  agents  can  work  towards  a  common  goal  in  a  complex,  real¬ 
time,  noisy,  collaborative,  and  adversarial  environment. 

Research  in  Sweden  has  developed  an  approach  for  directing  a  hierarchy  of  intelligent 
agents  in  real  time.  Their  approach  was  to  enable  control  of  UAVs  by  allowing  the 
human  controller  to  access  an  agent  at  any  level  of  abstraction,  as  needed.  The  human 
can  acquire  control  over  an  agent  in  a  hierarchy,  but  the  agent  will  be  expected  to 
continue  with  other  tasks  as  well  as  fulfilling  the  operator’s  request  (Scerri,  Reed,  and 
Torne,  1999).  This  architecture  avoids  the  common  problem  of  agents  abandoning 
tasks  when  interrupted  by  the  human  operator.  Agents  at  lower  levels  of  the  hierarchy 
negotiate  among  themselves  to  decide  which  will  be  responsible  for  a  task  given  by 
either  the  human  operator  or  a  higher-level  agent.  As  in  a  human  organization, 
guidance  and  requirements  can  be  issued  at  varying  levels  of  abstraction  and  carry 
different  weights. 

In  Australia,  researchers  have  developed  autonomous  mobile  robots  with  local 
intelligence.  The  robots  do  not  rely  on  a  Global  Positioning  System  (GPS),  but  use 
on-board  “vision”  sensors  to  perform  tasks  such  as  navigation,  map  generation,  and 
coordinated  group  behavior  (Petitt  and  Braunl,  2003).  The  authors  attribute  the  success 
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of  this  research  program  to  the  use  of  a  “behavior-based”  approach  to  controlling  the 
mobile  robot  agents,  rather  than  the  traditional  control  algorithm  approach. 

Under  the  DARPA  Mixed-Initiative  Control  of  Automa-teams  program  (MICA)  one  group 
of  researchers  applied  the  “Playbook”  concept  to  enable  a  single  human  operator  to 
supervise  a  team  of  six  robots  in  RoboFlag,  a  simulation  of  the  child’s  game  “capture  the 
flag”  (Parasuraman,  Galster,  and  Miller,  2003).  Emulating  a  child’s  game  may  seem 
trivial  until  one  appreciates  that  success  requires  teamwork,  perception  of  the  location  of 
the  high-value  target,  and  tactics  for  countering  opposition  offense  and  defense.  These 
capabilities  are  not  unlike  a  UAV  ISR  mission. 

Johnson  (2003)  described  the  DARPA  MICA  program  as  having  two  approaches  to  one 
goal — the  control  of  large-scale  teams  of  semiautonomous  vehicles  by  a  relatively  small 
number  of  human  operators.  The  two  approaches  are: 

1 )  autonomous  control  theoretic  techniques;  and 

2)  mixed-initiative  techniques  for  integrating  humans  into  the  control  process. 

The  first  approach  is  based  on  a  hierarchical  command  and  control  structure  including: 

•  Team  composition  and  tasking; 

•  Team  dynamics  and  tactics;  and 

•  Cooperative  path  planning. 

The  second  approach  focuses  on  how  to  design  the  system  around  the  potential 
behavior  of  the  human  operators: 

•  Meaningful  cooperation  between  human  decision  makers  and  teams  of 
semiautonomous  entities;  and 

•  How  is  the  performance  and  stability  of  the  system  affected  when  an  operator 
can  take  control  at  varying  levels  and  times? 

Johnson  (2003)  makes  the  important  point  that  “it  is  rare  historically  for  an  R&D  program 
to  explicitly  consider  human  interaction  issues  simultaneously  with  technology 
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development.”  The  DARPA  MICA  program  objectives  established  a  requirement  for 
robotics  and  control  engineers  to  work  collaboratively  with  cognitive  engineers  and  HSI 
experts  to  achieve  a  human-centered  design  perspective. 

The  MICA  program  was  discontinued  after  approximately  two  years  into  the  four-year 
plan.  According  to  the  coordinator  of  the  MICA  program,  the  progress  was  apparent 
during  the  first  two  years,  but  it  also  became  clear  that  both  the  Unmanned  Combat 
Armed  Rotorcraft  (UCAR)  and  Joint  Unmanned  Combat  Air  Systems  (J-UCAS) 
programs  would  have  to  pursue  developments  similar  to  MICA  in  multi-UAV  control. 
DARPA  concluded  that  further  development  of  this  technical  topic  should  be  continued 
within  the  scope  of  those  larger  and  pragmatically  focused  programs  (Kott,  2004). 

Autonomy 

A  key  issue  for  the  design  and  implementation  of  future  UAVs  is  “levels  of  autonomy,” 
which  can,  theoretically,  range  from  teleoperation  to  fully  autonomous  systems. 

Yavanai  (2003)  defined  autonomy  as, 

an  attribute  of  a  system  which  characterizes  its  capability  to  accomplish 
the  system’s  assigned  mission  goals  without  any,  or  with  only  minimal, 
abstract  level  intervention  of  an  external  cooperative  entity,  i.e.,  a  remote 
agent  or  a  remote  operator,  while  the  system  is  operating  under 
constraints  and  under  unstructured,  unexpected,  and  dynamic  uncertain 
environment  as  well  as  under  evolving  dynamic  uncertain  scenario 
conditions. 

Neidhoefer  and  Krishnakumar  (2001)  developed  an  intelligent  aircraft  control 
architecture  based  on  “levels  of  intelligent  control.”  They  adopted  concepts  from 
computational  intelligence  such  as  neural  networks,  genetic  algorithms,  and 
adaptive  critics. 

Robotics  research  programs  have  wrestled  with  ways  to  enable  multiple  levels  of 
autonomy.  Studies  at  the  Johnson  Space  Flight  Center  have  identified  the  need  for 
adjustable  autonomy  (AA)  because  humans  sometimes  need  to  be  involved,  not  only  at 
the  highest  (decision/deliberative)  level,  but  all  the  way  down  to  teleoperation  of  systems 
that  were  intended  to  be  fully  autonomous  (Bonasso,  1999).  Their  experience  in 
developing  space  applications  led  to  the  following  recommendation:  design  the  control 
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architecture  for  full  autonomy;  then  relax  the  autonomy  restriction  at  each  level, 
beginning  with  the  highest. 

The  Canadian  Defence  Research  and  Development  group  compiled  a  technology 
summary  of  autonomous  collaborative  unmanned  vehicles  (Bowen  and  MacKenzie, 
2003).  The  report  concludes  that  autonomous  control  is  the  key  to  achieving  maximum 
UAV  utility.  This  conclusion  extends  to  Unmanned  Vehicles  (UV)  of  all  kinds — space, 
air,  ground,  water  surface,  undersea,  and  other  types  of  vehicles.  Gaps  in  knowledge 
were  identified  in  the  following  technology  streams:  Robotics,  Mobility,  and  Navigation. 

Another  approach  is  to  consider  an  intelligent  robot  to  be  capable  of  acting  as  a  peer  of 
the  human  operator,  worthy  of  exhibiting  collaborative  control  (Fong,  Thorpe,  and  Baur, 
2001).  This  type  of  relationship  between  the  human  and  the  robot  also  is  called  a 
“mixed-initiative”  architecture. 

One  objective  is  to  develop  a  control  architecture  that  will  allow  a  single  human  operator 
to  interact  with  multiple  robots,  while  maintaining  reasonable  workload  and  effectiveness 
(Goodrich,  Olsen,  Crandall,  and  Palmer,  2001).  This  objective  can  be  extended  to  allow 
multiple  human  users  to  manage  multiple  robots,  perhaps  of  different  types,  from 
multiple  control  platforms,  some  of  which  may  be  mobile. 

Swarms  and  Stigmergy 

Several  concepts  merge  under  the  heading  of  “swarms”  including  biomimetics, 
stigmergy,  self-organizing  systems,  and  emergent  behavior.  These  concepts  are  worthy 
of  analysis  for  their  potential  application  to  UAV  design  and  operations.  “Biomimetics”  is 
the  term  given  to  biologically  inspired  technology  (Bar-Cohen  and  Breazeal,  2003). 
Biomimetic  programs  seek  to  simulate  or  mimic  the  mobility,  intelligent  operation,  and 
functionality  of  biological  creatures. 

One  aspect  of  biomimetics  is  the  analysis  of  swarming  behavior,  which  is  common  in 
nature  (e.g.,  bees,  wasps,  and  ants).  Similarly,  flocks  of  birds  and  schools  offish  exhibit 
closely  coordinated  group  behaviors  (Clough,  2003). 
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According  to  a  NASA/JPL  Website  rhttp://dsp.ipl.nasa.qov/members/payman/swarm/l. 

Swarm  Intelligence  (SI)  is  the  property  of  a  system  whereby  the  collective 
behaviors  of  (unsophisticated)  agents  interacting  locally  with  their 
environment  cause  coherent  functional  global  patterns  to  emerge.  SI 
provides  a  basis  by  which  it  is  possible  to  explore  collective  (or 
distributed)  problem  solving  without  centralized  control  or  the  provision  of 
a  global  model. 

The  rise  of  agent-based  simulation  systems  is  in  concert  with  swarming  biomimetics. 

Hoffmeyer  (1994)  defined  a  swarm  as,  “a  set  of  mobile  agents  which  are  liable  to 
communicate  with  each  other  directly  or  indirectly  and  which  collectively  carry  out  a 
distributed  problem  solving.”  An  individual  insect  has  extremely  limited  intelligence,  yet, 
by  following  simple  rules,  the  emergent  behavior  of  the  swarm  efficiently  gathers  food, 
transports/retrieves  heavy  prey,  and  defends  itself. 

Stigmergy  is  the  use  of  the  environment  to  communicate.  Bonabeau  and  Theraulaz, 
(2000)  discuss  how  ants  leave  a  trail  of  pheromones,  a  form  of  stigmergy,  to  forage 
efficiently.  For  example,  an  ant  leaves  a  pheromone  trail  outbound  and,  after  finding 
food,  doubles  the  intensity  of  the  pheromone  trail  when  inbound.  Subsequent  ants  will 
tend  to  follow  the  strongest  trail.  These  same  rules — the  pheromone  serves  as  an 
attractor  sign,  but  has  a  decay  function — can  be  used  computationally  to  solve  difficult 
problems  such  as  the  “traveling  salesman.”  In  this  problem,  a  person  must  find  the 
shortest  route  by  which  to  visit  a  given  number  of  cities,  each  only  once.  This  classic 
problem  is  “devilishly  difficult”  using  traditional  computational  methods  because  for  just 
15  cities,  there  are  billons  of  route  possibilities  (Bonabeau  and  Theraulaz,  2000).  For 
UAV  applications,  an  analogous  type  of  communication  among  the  “swarm”  could 
provide  information  about  what  areas  of  a  search  region  have  been  recently  covered. 

“Digital  pheromones”  have  been  suggested  by  Parunak,  Purcell,  and  O’Connell  (2002) 
as  an  effective  way  to  coordinate  swarming  UAVs.  They  describe  digital  pheromones  as 
analogous,  but  better  than  the  use  of  electrostatic  potential  fields  to  control  movement. 
New  information  is  quickly  integrated  into  the  field,  while  obsolete  information  is 
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automatically  forgotten  through  pheromone  evaporation.  After  developing  simulations  as 
proof  of  concept,  Parunak  et  al.  (2002)  concluded  that  “swarming  techniques  inspired  by 
insect  pheromones  offer  a  powerful  mechanism  for  coordinating  unmanned  vehicles 
such  as  UAVs.” 

Paul  Gaudiano  and  colleagues  at  Icosystem  Corp.  are  currently  working  under  contract 
to  DARPA  to  develop  control  strategies  for  robot  swarms.  Gerla  and  Yi  (2004)  from 
UCLA  are  sponsored  by  the  Office  of  Naval  Research  (ONR)  to  investigate  team 
multicast  communications  among  autonomous  sensor  swarms.  The  Air  Force  Institute 
of  Technology  has  developed  a  model  of  swarm-based,  networked,  sensor  systems  and 
espoused  measures  for  the  evaluation  of  swarm  performance  (Kadrovach  and  Lamont, 
2002). 

One  aspect  of  swarm  behavior,  seen  clearly  in  flocks  of  birds  and  schools  offish,  is  their 
ability  to  “see  and  avoid.”  That  is,  they  perceive  and  control  their  distance  to  their 
nearest  neighbors  and  adjust  altitude/depth  and  heading  accordingly.  How  can  this 
“swarming”  emergent  behavior  contribute  to  UAV  design?  One  fundamental  aspect  of 
“see  and  avoid”  is  to  avoid  one’s  own  teammates.  This  aspect  of  the  swarming  analog  is 
entirely  relevant  to  UAV  applications.  The  other  aspect  of  “see  and  avoid”  in  UAVs  is  to 
avoid  terrain  obstacles  and  all  other  aircraft  in  the  National  Air  Space.  It  remains  to  be 
seen  whether  those  issues  may  be  informed  by  swarming  emergent  behavior. 

Parunak,  Brueckner,  and  Odell  (2003)  investigated  three  main  approaches  to  swarming 
coordination: 

1 )  relationship  between  individual  agents  and  their  group  (“roles  analysis”); 

2)  optimizing  systems  in  light  of  constraints;  and 

3)  processes  inspired  by  natural  systems  (digital  pheromones  and  biomimetics). 

They  investigated  various  instantiations  of  these  approaches  by  creating  software 
models. 

Defence  R&D  Canada  compiled  a  review  of  swarming  UAVs  from  a  control  engineering 
perspective  (Kim,  Hubbard,  and  Necsulescu,  2003).  The  authors  describe  swarming 


40 


Human  Systems  Integration  and  Automation  Issues  in  SUAVs 


entities  as  autonomous  units  that  can  gather  from  different  locations,  act  together,  and 
then  disperse.  Swarming  entities  are  decentralized,  tolerant  to  variances  of  the  units,  or 
to  addition/deletion  of  units. 

Kim,  et  al.  (2003)  warn  that  the  specificity  of  fixed-wing  UAVs  must  not  be  ignored.  They 
are  not  omnidirectional  and  cannot  fly  below  a  certain  speed;  therefore,  group  dynamics 
solutions  must  take  into  account  the  aerodynamic  limitations  of  specific  fixed-wing  UAV 
flight  envelopes. 

Researchers  and  engineers  are  investigating  alternative  approaches  to  control 
trajectories  of  multiple  UAVs,  get  sensors  to  the  intended  search  locations,  and  avoid 
collisions.  Vincent  and  Rubin  (2004)  from  UCLA  reviewed  the  concept  of  cooperating 
swarms  and  report  on  the  design  and  analysis  of  cooperative  search  by  simulated  UAV 
swarms. 

Sigurd  and  How  (2003)  reviewed  local  approaches  to  swarming  based  on 
nearest-neighbor  interactions  or  potential  fields.  They  summarize  “flocking”  approaches 
based  on  far-field  attraction  and  a  near-field  repulsion  between  vehicles,  but  caution  that 
perfect  information  about  the  location  and  dynamics  of  each  vehicle  is  unlikely  for  a  UAV 
swarm.  Sigurd  and  How  (2003)  suggest  the  application  of  a  “total  field”  approach  using 
magnetic  fields,  and  indicate  that  some  biologists  believe  that  similar  mechanisms  may 
be  found  in  natural  systems  such  as  flocks  of  birds  and  schools  of  fish.  Their  results 
suggest  that  safe,  but  aggressive,  navigation  can  be  supported  by  this  magnetic  dipole 
approach  without  requiring  each  vehicle  to  know  the  position  of  any  of  the  other  vehicles. 

The  Navy  program  called  “SWARM  UAV”  is  only  loosely  related  to  the  biomimetic 
concepts  that  inspired  the  acronym — Smart  Warfighting  Array  of  Reconfigurable 
Modules.  This  program  was  focused  on  the  aeronautical  development  of  a  small, 
inexpensive  UAV  that  could  be  used  for  a  variety  of  purposes,  including  possible  swarm 
applications  (Castelli  and  Howe,  1999).  This  SUAV  later  became  known  as  Silver  Fox. 

Figure  5  provides  a  schematic  contrast  of  the  control  links  and  the  associated  workload 
or  cognitive  demand  placed  on  a  human  controller  for  conventional  control  of  multiple 
UAVs  and  for  swarm  control. 
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Figure  5.  Control  types  of  multiple  UAVs  (the  thicker  the  communicating  lines,  the  heavier 
the  workload  and  cognitive  demands  on  the  CGS  operators). 

Entirely  autonomous,  emergent  behavior  is  not  a  reasonable  goal  for  military  UAV 
systems.  But,  incorporating  some  aspects  of  effective  swarm  behavior  in  combination 
with  top-level  human  control  (see  the  “Playbook”  example  below)  may  prove  to  be  an 
effective  approach  to  achieving  a  favorable  ratio  of  UAVs  to  humans  and  accomplishing 
ISR  missions  with  minimal  risk  to  humans. 

Semiautonomy  and  the  Human-Robot  Interface 

In  semiautonomous  systems,  a  human  controller  acts  in  concert  with  an  intelligent 
system  (semiautonomous  agent).  The  level  of  autonomy  may  span  a  large  range  and  it 
may  be  either  constant  or  adjustable,  varying  with  time,  task  difficulty,  scenario  segment, 
or  the  status/health/workload  of  the  human.  Designing  the  logical  and  the  physical 
interfaces  between  the  human(s)  and  the  intelligent  system  is  an  important  task.  This 
interface  is  beginning  to  be  known  as  the  Human-Robot  Interface  (HRI). 

According  to  Olsen  and  Wood  (2004),  HRI  differs  from  traditional  computer-human 
interface  (CHI)  design  in  two  key  ways: 

1 )  robots  operate  in  a  physical  world  that  is  not  completely  under  software  control; 
and 

2)  the  physical  environment  encountered  by  the  robot  imposes  its  own  forces, 
timing  and  unexpected  events  that  must  be  dealt  with  by  the  HRI  system. 
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One  common  goal  of  HRI  is  to  enable  one  human  controller  to  control  multiple  robots. 
The  number  of  robots  that  can  be  operated  is  called  the  “fan-out”  of  a  human-robot  team 
(Olsen  and  Wood,  2004).  Goodrich,  Olsen,  Crandall,  and  Palmer  (2001)  offer  the 
following  classification  for  research  on  HRI: 

1)  autonomous  robots; 

2)  teleoperation; 

3)  adjustable  autonomy; 

4)  mixed  initiatives;  and 

5)  advanced  interfaces. 

Work  on  teleoperated  systems  is  the  most  mature  (Sheridan,  1992),  but  a  difficult 
obstacle  to  effective  teleoperation  is  the  time  delay  in  communications  incurred  as  a 
function  of  the  distance  between  the  human  operator  and  the  robot.  This  is  clearly  an 
important  issue  for  some  UAV  operations.  In  teleoperated  systems,  two  approaches  to 
this  problem  have  been  taken — the  use  of  quickened  or  predictor  displays  and  the 
application  of  automation  features  combined  with  supervisory  control. 

Research  on  the  human  role  in  supervising  robots  or  semiautonomous  systems  has 
revealed  both  benefits  and  costs  of  various  approaches  and  HRI  designs.  Systems  that 
reduce  human  workload  by  participating  in  higher-level  decision-making  tasks  are 
effective  only  if  the  automation  is  entirely  reliable,  which  is  a  difficult  undertaking 
(Parasuraman,  Galster,  and  Miller,  2003).  Consequently,  the  current  view  is  that  the 
best  approach  to  HRI  and  to  the  interaction  between  humans  and  automated  systems  in 
general,  is  that  the  interface  should  be  adjustable  or  adaptive.  Humans  should  be  able 
to  delegate  tasks  as  needed,  and  be  provided  with  feedback  information  supporting 
supervisory  control  of  those  tasks.  According  to  Parasuraman  et  al.  (2003),  this  is 
equivalent  to  delegation  as  practiced  in  successful  human  teams. 

The  term  “Optionally  Piloted  Vehicle”  has  been  used  to  convey  the  concept  of  adjustable 
autonomy.  The  human  controller  has  the  option  to  take  control  of  the  vehicle,  reverting 
from  an  autonomous  or  shared  responsibility  to  a  Remotely  Piloted  Vehicle,  i.e.,  fully 
manual.  The  transition  or  transfer  of  control  is  the  challenging  part  of  the  system  design. 
Designing  a  control  architecture  and  user  interface  that  enables  graceful  transitions  in 
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control  in  both  directions — to  and  from  human  controller(s)  to  semiautomated  agents — is 
an  enormous  challenge,  but  one  that  has  substantial  potential  benefits  for  UAV 
operations. 

Christopher  Miller,  in  conjunction  with  Honeywell  and  SMA  Information  Flow 
Technologies,  seems  to  have  coined  the  term  “Playbook”  (Miller,  Funk,  Goldman,  and 
Wu,  2004).  Using  the  sports  metaphor,  Playbook  represents  a  hierarchical  system 
where  the  human,  operating  as  the  quarterback  or  coach,  can  “call  the  play”  and  the 
autonomous  agents  will  execute  their  roles  in  the  play.  In  mixed-initiative  systems, 
presumably,  one  or  more  of  the  players  could  suggest  plays  to  the  decision-maker.  The 
Playbook  concept  is  consistent  with  earlier  work  on  dynamic  function  allocation,  also 
called  “adaptive  automation,”  in  which  human  and  automated  tasks  can  be  allocated 
dynamically,  depending  on  operator  choice,  workload,  fitness,  or  similar  variables 
(Kantowitz  and  Sorkin,  1987;  Morrison  and  Gluckman,  1994). 

MACBETH  is  a  tactical  planning  software  tool  designed  for  applications  in  which  a 
human  user  must  quickly  specify  a  mission  to  a  team  of  autonomous  agents.  MACBETH 
combines  hierarchical  task  network  planning  with  constraint  reasoning  into  a 
mixed-initiative  planning  system  consistent  with  the  “playbook”  metaphor  (Goldman, 
Haigh,  Musliner,  and  Pelican,  2000). 

The  Playbook  concept  deserves  further  research  and  development.  It  provides  an 
excellent  conceptual  architecture  for  human  control  of  a  multiple-UAV  system. 
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